Saturday, 1 March 2014

Using Google Datastore, Google App Engine, and Endpoints for your applications

Google Datastore is a very convenient cloud based database to use in your mobile or web applications. However, the process of integrating it with the application is somewhat complex, and the documentation is not very helpful.

I am presenting a step-wise procedure of how we used Google Datastore with DrawTyme applications. In this post, only the integration with web-app of DrawTyme is explained. Integration with the Android app will be the subject of the next post.

Background 

Google Datastore a non-relational database. It is very simple and inexpensive to use. You need to use Google App Engine and Google Endpoints to access it.

For the purpose of this tutorial, each data entry in the database, will be of type Video. Each Video has the following components
  1. ID
  2. Owner
  3. Title
  4. Author
  5. Description
  6. Timestamp of creation
This data is sufficient to retrieve a video from the cloud storage.

 

The Procedure

The following steps need to be taken in order to use Google Datastore.

1. Setup Development Environment
2. Create a project on Google App Engine 
3. Download and modify the Java Backend and Javascript Web-App code
4. Upload the Backend Code to Google App Engine

 

Setting up the development enviroment

  1. Download and install "Eclipse IDE for Java EE Developers" from Eclipse Download Site. (Make sure you have Eclipse IDE for Java EE Developers. My experience was that Google Plugin for Eclipse does not work well with other flavours of Eclipse)
  2. Install "Google Plugin for Eclipse". You can find instructions at the Google Plugin for Eclipse site.

Creating a project on Google App Engine

  1. Follow the instructions for "Creating an App Engine app" at https://developers.google.com/appengine/docs/java/endpoints/getstarted/backend/setup
  2. Copy the project ID of the project you just created

Download the backend code

  1. Start Eclipse and create a Web Application as described in "Creating your first Web Application" section of the Google Plugin for Eclipse site.
    • A directory for this project will be created in your workspace
  2. Download DrawTymeBackendExample.zip and extract it's contents into the directory created in #1 above.

Understanding the code

First the Java Backend:
  1. the "src" folder 
    1. The file Ids.java declares the Client IDs that are authorised to use the app engine
      • To generate these IDs, open https://cloud.google.com/console
      • Open the project you created earlier
      • On the left hand menu, click on APIs & auth
      • Then click Credentials
      • Create a new OAuth ID for each web/Android/iOS application that you want to allow use of theis App Engine project
    2. The file VideosV1.java defines the api that will be called by our apps
  2. the "war" folder
    1. the file WEB-INF/appengine-web.xml
      • In this file, write the application ID of the project you created on Google App Engine, in the <application></application> tag
    2. the file WEB-INF/datastore-indexes.xml
      • The contents of this file were suggested by Google App Engine, and I got it right after a few iterations
Now the Javascript code:
  1. war/js/backend.js
    • This file has functions that call the api of the Backend
    • Put the Client ID of the web-app that you created and wrote in the Ids.java file above 
  2. war/index.html
    1. This is left blank - you can write your own code here to test out the functions in the Backend. The API required for these functions is initialised and installed in war/js/backend.js

Uploading the Backend Code to Google App Engine

  1. Follow the instructions on "Deploying your Web Application" at https://developers.google.com/eclipse/docs/getting_started#deploying
And we are done! Once deployed, your project is now available at http://application-id.appspot.com/ where application-id is the project ID of your project on Google App Engine. 
(IMPORTANT: Make sure to use https:// rather than http:// when accessing your project. Otherwise, the user signin - which uses OAuth2 - will not work)
I have tried to keep this as simple as possible. If you have any queries, please feel free to mail me at info[at]drawtyme.com, and I will share what I know.

13 comments:

  1. Thank you.. It was helpful..
    Eagerly waiting for your post on integration with Android App..

    ReplyDelete
    Replies
    1. Hi!

      Thank you for your feedback! Such feedback is very encouraging!

      I have been really short on time. However, I'll post the part 2 very soon.

      Delete
  2. Thanks a lot for great tutorial. I think got what you explained, but still have a question:
    I added to index.html input fields for authorName, title, and description for Video object. How to call now Endpoint videos.insert()? How from HTML call videos.list()?
    Please explain or show an example.
    Thank you for your time.

    ReplyDelete
    Replies
    1. Hi!

      videos.insert() and videos.list() are javascript functions, so to call them, you need to create buttons, and in the "onclick" parameter, call the corresponding functions from backend.js (I am assuming you have downloaded the zip file mentioned in the blog post)

      Remember, though, that videos.insert() is an authenticated method, so the user has to signin before successfully inserting a video.

      In order to signin, you need to have a signin button, and then call the myapi.example.signin(mode) function from backend.js. Make sure you have set the CLIENT_ID in backend.js correctly

      mode is a boolean - either true or false. In this case, it should be false.

      Let me know whether this works for you.

      Delete
    2. Dear Gaurang,
      Thanks a lot for prompt response. This is my very simple code that does not work:
      index.html
      script:
      function mysubmit() {
      var name = document.getElementById("name").value;
      var title = document.getElementById("title").value;
      var descr = document.getElementById("descr").value;
      myapi.example.insert(title, author, desc);
      }

      body:

      Author Name: input type="text" id="name
      Title: input type="text" id="title"
      Description: input type="text" id="descr"
      (button onclick="mysubmit()") Submit(/button)
      (button onclick="myapi.example.list(1);")List(/button)

      Sorry for syntax, HTML tags not allowed here.
      Both buttons does not do anything. I do not see any message on console message. I followed your tutorial, created Client Id and placed in Ids.jva WEB_CLIENT_ID and to backend.js myapi.example.CLIENT_ID. Only thing that I didn't do that after deployment link "http://application-id.appspot.com/" lead me to the wrong site. But in my understanding it is not important.
      Thank you so much for your help!

      Delete
    3. I think your code is fine, except there is a spelling mistake when you call
      myapi.example.insert(title, author, desc);

      desc should be descr.

      Apart from that, in the url - http://application-id.appspot.com/ - replace appication-id by the Application ID of your project that you created on App Engine. (BTW, this URL is very important, that's where your application is available to the outside world!)

      Make sure you use https, rather than http, since signing in works only over https.

      You can easily find out what the response of the server is, when you make the api calls. In backend.js, for the api call functions myapi.example.list(), etc, there is a "resp" object, which contains the response of the server. You can JSON.stringify() it, and print it to console using console.info()

      Hope this helps!

      Delete
    4. could you share the code for index page. i am unable to move forward.please help me out

      Delete
    5. Dear,
      Thank you so much for your time and help. I appreciate it a lot. But I still need your help. I have 2 problems, one local run and one on deployed:
      1. When I am running locally: my java script calling myapi.example.insert(title, name, descry); (backend.js),
      at the statement gapi.client.myapi.videos.insert({…}) I am catching error: "TypeError: Unable to get property ‘videos’ of undefined or null reference."
      Could it be because I don’t see anywhere in the project reference to myapi-v1.api or I am missing something?

      2. When I upload backend code to GAE, I am getting error:
      Request URL: https://mobileroadwarrior3.appspot.com/_ah/api/rpc
      Method: myapi.videos.insert
      Error Code: 401
      Reason: required
      Message: com.google.appengine.api.oauth.OAuthRequestException: Invalid user.

      I created Client ID that looks like “6…6.apps.googleusercontent.com” and I placed this value to
      Ids.java: public static final String WEB_CLIENT_ID = "6…6.apps.googleusercontent.com";
      backend.js: myapi.example.CLIENT_ID = '6…6.apps.googleusercontent.com';

      What I am doing wrong?

      Thank you.

      Delete
    6. Hello Rimma,

      For your local version, the gapi.client is not loaded, since it requires that oauth api be loaded. This can't happen on your localhost, so that function is never completed successfully. Either don't load oauth api in the init() function, or else, don't bother about testing locally - in any case, it's not required.

      For your deployed version, you need to also provide a signin button, as I mentioned in my previous reply. Without the signin, the insert() will fail, since it's an authenticated method. Also, you haven't provided a callback to the insert(). Without the callback, the function won't know what to do after it returns.

      Bests,
      Gaurang

      Delete
    7. Hello Ponduri,

      Have you used the index.html file in the code mentioned in this blog?

      To the <head> of that file, add

      <script>
      function initialise() {
      myapi.example.init('https://' + window.location.host + '/_ah/api');
      }

      function insert() {
      myapi.example.insert(document.getElementById("title").value, document.getElementById("name").value, document.getElementById("desc").value, onInsert);
      }

      function onInsert(resp){
      console.info("insert successful: "+JSON.stringify(resp));
      }

      function list() {
      myapi.example.list('1',onList);
      }

      function onList(resp){
      console.info("list successful: "+JSON.stringify(resp));
      }
      </script>


      then, to the <body>, add the following:

      <button onclick="myapi.example.signin(false);">Sign In to Google</button>
      <hr>

      <label for="name">Author Name: <label><input type="text" name="name" id="name"><br/>

      <label for="title">Title: <label><input type="text" name="title" id="title"><br/>

      <label for="desc">Description: <label><input type="text" name="desc" id="desc"><br/>

      <button onclick="insert();">Insert</button>
      <button onclick="list();">List</button>


      Hope this helps!

      Best,
      Gaurang

      Delete
  3. Dear Gaurang,
    Thanks a lot, this example works for me !
    Thank you for your help!

    ReplyDelete