Saturday, November 29, 2014

Publishing a map with Map Engine

I have used a number of mapping programs to create routes. They are route based programs. I have also used "My Map" (Google) to create maps with routes and other information. The most notable one  is PCN on Bikes. It was later recreated into the Map Engine Lite as New PCN on Bike.

The former has been migrated to the latter. That caused a problem. There is a maximum of 5 layers and no Custom Icons. Also the uploading is limited to 500 items per layer. I can't just import and export the whole map in one go and I can't upload the old PCN on Bike onto the new. It took me quite some time to redo the whole thing in the LITE version. The result is a map that uses default icons and just 5 layers. I is not pleasant to look at. Thus it is important for me to look for alternatives.

One alternative is to pay $5 per month for the PRO version. However, the limits is just extended to 10 Layers and 2000 items. It probably served all my purposes but  I refused to pay for it. I am not running a business on the map and gains nothing from it besides the praise from uses. One friend offers to sponsor me. I am hesitant to accept it.

One other option is to use the Map Engine evaluation. It is something like PRO but with 80,000 load traffic and 10 MB data size limits. It allows custom icons and large number of layers. This looked good.

It is a hard time trying to use Map Engine though. Firstly it does not have editing feature. Luckily it can import KML files. Thus I imported the KML from the Map Engine Lite and after a long and tedious process of uploading to the Data Source, adding to Layer and then adding to Map. I managed to create a seeable map. I thought that will be the end of the story.

On helping another group to create a map, I realized that there is an app in Android called "My Maps". It can load the maps onto the hand phone and view it on the go. That is great news till I viewed it. The map cannot be loaded reason being that the app cannot load map with KML layers. That is a huge let down. If it can be loaded then surely it should be displayed.

I guess Google does not want Google Engine to be use by the Map Engine Lite people to by pass the custom icon and layer limitation settings. If it can be easily imported into Map Engine then they will loss considerable business.

I am then left with no choice but to find alternatives. From the upload dialog, I find that it accepts uploads in shapefile format (SHP). A SHP file is basically a vector file. Thus I must convert the KML to shapefile first. Now, how do I do it? I simply have no idea. There are a few converters that is available to convert KML to SHP bu tthey all fall short of expectation. In fact, I don't know that the Map Engine only accepts points (markers), lines and polygons. Thus all I get is lines and round dots for the markers after conversion.

An effort is thus done to find a vector editing file and one that could import KML files. QGIS appears in the search. I downloaded it and installed. On first run it pops up a dialog to define map projection definition.

I was dumbfound. What in the world is that? I simply have no idea as it is the first time I use a GIS (Geographic Information System) editor. I edited OpenStreetMap before and it is a web based one. It is just a graphic editing system which draw lines and polygons. It does not even require any definition and I have no idea what format it stores the data.

Searching internet, I finally found out what is it all about. I also find out what Google uses in its maps. It uses WGS64 format. For example a coordinate reference to a point in singapre is Lat 1.26503 Long 103.82193 (Vivo MRT). I have been using this in OSMAnd (Android Map app)  without even knowing it.

With the CRS set, I was then prompted to select a location. It showed the world map. Again I was intimidated. I cannot zoom into the map and select an area. The only way is to select from a list and Singapore is simply not in the list. It ended up I choose Malaysia and then add a bit of offset to it to ensure Singapore is covered.

The Application appeared after a long wait. It was a blank work space with a lot of toolbars and panels. Third time whammy. How to draw a line when you don't even know where to put it?

After looking at the menus I finally find an item called "Vector" in it there is a item called "OpenStreetMap". I thought that is it. So I click "Download data". After the data is downloaded, I am still looking at a blank screen. On searching internet again. I found out that I had to "Import topology from XML" then "Export topology to spatialLite". I don't know what in the world is that but followed faithfully to do three downloads for each type (line, point, polygon).

Well, it is not surprising that I get a blank area again. This time there are three items appearing on the left panel. I left click and find an item called "zoom to layer". A wire mess appeared with large patches of green. Finally I get something. On zooming in I can see Malaysia. Zooming in further, I finally get to see Singapore. It was covered with green and a huge mess of wires but at least I can recognize that it is Singapore roads and buildings. On playing further I managed to turned off the points and polygon thus leaving less clustered screen. Well it is not what I expected but it will do.

The next step is then to import the KML. That is pretty easy. There were instructions in the internet. So I imported the maps from my map. Finally I get to see my map and the not so nice base map of OSM.

The import looked ok except the Notes layer. It is just a bunch of circles. I managed to get the line layers to show its lines with a different color using styles. But struggled to define the Notes. There is no individual style setting. The Categorize style also don't allow selection of icons. I ended up choosing "rule based". I was tough trying to set the rules as I did not organize the Notes in the rules style. After a lot of trial and error, I managed to get it done. I am yet to use custom icons at this stage.

The next step is to load custom icons. It is again a challenge to do it. But via internet I managed to get it done with SVG icons. I learned SVG coding before thus it is easy for me to create the custom icons. In actual fact the icons in the first version of Google map was originally created using SVG.

It is the first time I feel that at least there is something familiar between QGIS and Google Map. The happiness was short lived. After I upload the vector maps into Map Engine. I get the same mess as when I converted the KML in QGIS. All the settings in QGIS is missing. There are also waning that some lines exceed the 50 vertex limit. I simply let go and take a break.

After thinking over whether to continue with the fiasco, I decided to push on. It is never my habit to give up so easily. I redo every style in Map Engine. It is yet another learning curve for me. When coming to style the Notes, I am faced with another challenge. There is no SVG icons. So I converted all the SVG again to PNG. But the conversion does not make the background transparent. At least I don't know how to do it in Inkscape (a SVG editor). Well that does not deter me. The graphic viewer in my computer has the ability to convert graphic formats. I simply use it to make GIF files with transparent backgrounds. It worked just fine. Now I have custom icons.

The next thing to do is to break the lines so that it is within the limits. Gosh! That is some thing I am yet again struck with unbelief. The layers simply cannot be edited. After a while I managed to toggle the editing feature. It is a safeguard system in QGIS. The next question is then how to break the lines into two. Even Map Engine does not have that feature. Well, at least I am lucky in this. There is a feature to break the lines. It is hard to use in the first time round but I managed to get it working.

On uploading to Map Engine, I discover that it does not replace the old resource. It simply creates another one. I really vomit blood on that. Map Engine is just not user friendly. I had to unpublish the old map, unpublish the old layer then unpublish the resource to remove the old one then reverse the process to add the new one. In the process the style gets lost. I almost utter the Fxxx word. Fortunately there is a "import rules from another layer" feature. I really heave a sign of relief. I am yet to try it on Notes but I am confident it can be done.

While I was trying out, a friend wrote to me about some errors in the PCN layer. It has some lines overlapping the Others layer. So I tried to edit it in QGIS. Well, that poses a problem. I just don't know how to edit besides deleting, moving the vertex (the joins in the line) and breaking the line. There is simply no facility to do that. Well, there is a plugin feature. So I tried to look at the long list of plugins and then I discover that Google has a plugin for QGIS. Wow! that is something. Unfortunately, I had to learn how to use it.

After a lot of trial and error again. I managed to make the connection. Now uploading to MapEngine is a wee bit easier. It is just 3 clicks away.Cool. The uncool part is that it still does not override the old uploads. Well, at least I can live with it.

Actually, that was not what I originally wanted to do when I look at the plugins. I get carried away. So, it is back to the search. I managed to get a few plugins that deals with editing. The first one is to join to features (lines). If I break it, I must be able to join it. The other one is to extend the line. It is not as easy as Map Engine editing where you click on a segment of a line and you can add a new vertex but at least I could combine the breaking, extending and joining to achieve what is just one simple step in Google. Hopefully I don't need to add a lot of vertexes.

Well, with all that,  I am at least able to use Map Engine to publish maps with custom icons and multiple layers just like the original map. It will be hard work but at least it can be done.

There was a little surprise I get when I tried to learn more about QGIS. I discovered that the base map can be loaded easily by the menu item "web". I never click on it as I though it might be some links to the internet. In actual fact, it can load a number of different base maps beside Google map. The previous attempt to load OpenStreetMap is simply for the wrong purpose. Well, I am dumbfound.

There is one more thing I had to do. You see, Map Engine pose a page loading limit per month. I had to get my map to be free of the limit by putting it in the Google Map Gallery. Some of my maps automatically gets published in there but not my Map Engine map. Fortunately, there is a link to do it in the map engine itself. So I just submit it by filling a form and in a day's time it was approved. Hopefully Google will remove such mundane limit in future. I do understand that Google need to make a profit. There are some idiots like me who simply want to contribute to the society free of charge. After all, I gets nothing from the work beside the "thanks" from users.

For a person who know nuts about GIS, the attempt is very foolish to some. To me it is just some challenge which I take on to do what I wanted to achieve. It may have a lot of fumbling and mistakes without proper training but since when I did proper training for most of the things I do before starting. My next blog will show you how many times I venture into something totally new without any training.

Latest update. Google Map Engine is no longer available. After all the hard work and it is no longer available in less than three months. Its really a waste of time and effort.