Map Maker top contributors survey summary
253 views
Skip to first unread message
Jayanth Mysore
Jun 13, 2012, 12:32:33 PM6/13/12
to mapmaker-an...@googlegroups.com
Dear users -
My name is Jayanth and I am a Product Manager of Google Map Maker.
Recently, your RER and Advocate friends were kind enough to answer a few questions to help us gauge what issues mappers generally face. As both Advocates and RERs contribute thousands of quality edits and reviews, they see issues across a range of contexts that help us get a good sense for the product’s health. As these issues affect the entire Map Maker community, I wanted to share a summary of the learnings and actions that we intend to take to make mapping a better experience for us all.
Why did we run the survey?
Primarily, we wished to understand how mapper behavior is reflected by our current statistics. Statistically, we observed that in aggregate there was a decline in usage of Map Maker that did not triangulate with the level of engagement we were seeing from all of you on forums, monthly meet-ups, user conferences,...or, for that matter, the increase in usage from new users.
While we are very aware that our product does suffer from many drawbacks, we could not explain any significant regression that would cause a drop-off in volumes. So we wanted to ask questions that would help us test various hypotheses we had for the decline. Primarily we were looking to see if there was a sharp increase in concerns around several of our most commonly reported issues relating to the product, Google reviewers, bots etc.
Who took the survey?
All Advocates and RERs were invited. 100% of the Advocates and 50% of the RERs took the survey. In all, around 350 mappers took the survey.
What were the major findings?
Ok...now what does this mean and what are we doing about it?
1. We are going to invest in tools to make the review process more streamlined and efficient. This has been something we have been working for a while on and will get them out soon. I think there is also a need for a lightweight review tool - something that allows a local expert to factually assert the correctness of, instead of doing more in-depth analysis and taking judgement calls. The recent feature to enable "auto-denial after 7 days" is a step we have taken in this direction. There is lot more to come.
2. We have been spending a lot of time figuring out why edits are not appearing on Maps. We have identified at least one reason and soon you will see some changes to the workflow in Map Maker that should alleviate this issue. But this one is a multi-faceted issue and our investigation is still ongoing to find other causes and then fix.
3. With respect to what gets perceived as "incorrect judgement" by a Google Reviewer, we found one major issue with the way review outcomes were made. In cases where a user makes an edit and then a Google Reviewer marginally improves it (perhaps to comply with Google policies), we identified situations when this was incorrectly reflected as "denied". We will fix this ASAP. This issue (incorrect judgement by a GR) is again not a simple problem. We are investigating ways to better align how various teams work to give our users a more consistent and correct experience (those of you familiar with editing Google Place Pages and Map Maker have already speculated on these inconsistencies in the forum)
Fine...so you ask what are you doing about those really frustrating bugs?
You have to trust me on this, but what we have started doing is to set aside time every month to fix issues that have dogged the product forever. I'll try to share as much as I can. Slowly, but surely, you will see the benefits of these fixit-days, as we call them in our team. For example, we have identified and fixed almost all occurrences of the annoying “cannot merge batch edits” message. We also have made loading of the application faster by addressing a few issues (not claiming this one is fully solved at this point).
As users of the product, we experience these issues ourselves as well. Some of these issues remain because they arise due to fundamental design decisions made years ago when the assumptions around the product and the Google Maps ecosystem were very different. Map Maker has gone from being a system to enable mapping of places that did not have a map on Google, to becoming a "often instant" (fair? :) ) editor for Google Maps in places where a rich map already exists. With that increase in our impact comes a far more complex ecosystem with which we need to integrate. Almost every good change you see in Google Maps implies an infrastructure change for something in Map Maker to ensure nothing breaks - unfortunately this means you may not see "new features" or bugs getting fixed as often. Sometimes these changes in Maps also manifest as constraints to what we can do on Map Maker - I'll leave it to your creative imagination to guess these. ;)
The net summary is that we are making a commitment to fix them - even if it is going to happen days at a time, it is like exercise...the results will come as long as the commitment from us is there...and we are committed.
Thank you for your patience and the opportunity to call you all users of my product.
Sincerely,
Jayanth
on behalf of the entire Map Maker team
My name is Jayanth and I am a Product Manager of Google Map Maker.
Recently, your RER and Advocate friends were kind enough to answer a few questions to help us gauge what issues mappers generally face. As both Advocates and RERs contribute thousands of quality edits and reviews, they see issues across a range of contexts that help us get a good sense for the product’s health. As these issues affect the entire Map Maker community, I wanted to share a summary of the learnings and actions that we intend to take to make mapping a better experience for us all.
Why did we run the survey?
Primarily, we wished to understand how mapper behavior is reflected by our current statistics. Statistically, we observed that in aggregate there was a decline in usage of Map Maker that did not triangulate with the level of engagement we were seeing from all of you on forums, monthly meet-ups, user conferences,...or, for that matter, the increase in usage from new users.
While we are very aware that our product does suffer from many drawbacks, we could not explain any significant regression that would cause a drop-off in volumes. So we wanted to ask questions that would help us test various hypotheses we had for the decline. Primarily we were looking to see if there was a sharp increase in concerns around several of our most commonly reported issues relating to the product, Google reviewers, bots etc.
Who took the survey?
All Advocates and RERs were invited. 100% of the Advocates and 50% of the RERs took the survey. In all, around 350 mappers took the survey.
What were the major findings?
- A good 62% said that time spent in reviewing edits has taken time away from editing.
- Some of this is to be expected as all of our RERs got here because they are prolific mappers. On the other hand, if mapping a lot is something you enjoy, this indicates that we need to find ways to making reviewing less time consuming.
- Of those who said reviewing is taking a lot of time, 65% reported that quicker ways to review will help a lot, while the rest believe that "concealing edits where your reviews don't seem to make a difference" will be more useful.
- On the question of how the level of usage of Map Maker has changed in the last 3 months, 38% said that they used it less, while the remaining 62% used it either the same or more (roughly equal).
- We had follow up questions to understand each group better
- Those who map less chose roughly equally between the following as primary reasons:
- Edits not appearing on Google Maps
- Incorrect reviews
- Edits made by Google reviewer
- Those who map more chose roughly equally between
- Faster review of my edits
- More edits to review by other users
- More free time
- In terms of the free form feedback that we got, we heard the following big themes:
- Fix the bugs !
- Google reviewers are acting like they know my neighborhood better than me
- Bots are crazy. Get them out of Map Maker
- Rendering bugs of all sizes, flavors and colors
- RERs want equal impact of their moderation decisions across edits they review
Ok...now what does this mean and what are we doing about it?
1. We are going to invest in tools to make the review process more streamlined and efficient. This has been something we have been working for a while on and will get them out soon. I think there is also a need for a lightweight review tool - something that allows a local expert to factually assert the correctness of, instead of doing more in-depth analysis and taking judgement calls. The recent feature to enable "auto-denial after 7 days" is a step we have taken in this direction. There is lot more to come.
2. We have been spending a lot of time figuring out why edits are not appearing on Maps. We have identified at least one reason and soon you will see some changes to the workflow in Map Maker that should alleviate this issue. But this one is a multi-faceted issue and our investigation is still ongoing to find other causes and then fix.
3. With respect to what gets perceived as "incorrect judgement" by a Google Reviewer, we found one major issue with the way review outcomes were made. In cases where a user makes an edit and then a Google Reviewer marginally improves it (perhaps to comply with Google policies), we identified situations when this was incorrectly reflected as "denied". We will fix this ASAP. This issue (incorrect judgement by a GR) is again not a simple problem. We are investigating ways to better align how various teams work to give our users a more consistent and correct experience (those of you familiar with editing Google Place Pages and Map Maker have already speculated on these inconsistencies in the forum)
Fine...so you ask what are you doing about those really frustrating bugs?
You have to trust me on this, but what we have started doing is to set aside time every month to fix issues that have dogged the product forever. I'll try to share as much as I can. Slowly, but surely, you will see the benefits of these fixit-days, as we call them in our team. For example, we have identified and fixed almost all occurrences of the annoying “cannot merge batch edits” message. We also have made loading of the application faster by addressing a few issues (not claiming this one is fully solved at this point).
As users of the product, we experience these issues ourselves as well. Some of these issues remain because they arise due to fundamental design decisions made years ago when the assumptions around the product and the Google Maps ecosystem were very different. Map Maker has gone from being a system to enable mapping of places that did not have a map on Google, to becoming a "often instant" (fair? :) ) editor for Google Maps in places where a rich map already exists. With that increase in our impact comes a far more complex ecosystem with which we need to integrate. Almost every good change you see in Google Maps implies an infrastructure change for something in Map Maker to ensure nothing breaks - unfortunately this means you may not see "new features" or bugs getting fixed as often. Sometimes these changes in Maps also manifest as constraints to what we can do on Map Maker - I'll leave it to your creative imagination to guess these. ;)
The net summary is that we are making a commitment to fix them - even if it is going to happen days at a time, it is like exercise...the results will come as long as the commitment from us is there...and we are committed.
Thank you for your patience and the opportunity to call you all users of my product.
Sincerely,
Jayanth
on behalf of the entire Map Maker team
Reply all
Reply to author
Forward
0 new messages