Google sitempamps in Play?

[MS]

How do you guys create sitemaps in your Play apps?
Google sitemaps in particular. I'm sure many of you has resolved that
problem.
Lift framework has a sitemap generation module.
It would be pretty useful to have such a thing in Play.

[Chaitanya Sharma]

MS, any updates on this ?

--
You received this message because you are subscribed to the Google Groups "play-framework" group.
To post to this group, send email to play-fr...@googlegroups.com.
To unsubscribe from this group, send email to play-framewor...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/play-framework?hl=en.

[Guillaume Bort]

Should be pretty easy to achieve by extracting the information from
the Router object. Loop over Router.routes

--
Guillaume Bort, http://guillaume.bort.fr

For anything work-related, use g...@zenexity.fr; for everything else,
write guillau...@gmail.com

[Steren Giannini]

Hi,
At beansight.com, we are starting to need a sitemap generator. We intend to build a module for this.

Has someone already coded it? (If so, let's build something together :)
Do you have some advice or particular ideas about the implementation of this module?

Guillaume answered in a previous message:

Should be pretty easy to achieve by extracting the information from
the Router object. Loop over Router.routes

Thanks
Steren
http://www.beansight.com

[Dirk]

I've written some code to render sitemaps, although not as part of a module. It's included below.
One gotcha: yahoo's size limit for sitemaps is smaller than google or bing, so you may need to gzip larger site maps. This is probably best done by the webserver rather than a module, but maybe it can be included in the module documentation.
Going over the routes file is a good start, however presumably each of the main urls will have a different priority.

This is what I have in my conf:
sitemap.sites=google,yahoo,bing
sitemap.sites.google=http://www.google.com/webmasters/sitemaps/ping?sitemap=
sitemap.sites.yahoo=http://search.yahooapis.com/SiteExplorerService/V1/ping?sitemap=
sitemap.sites.bing=http://www.bing.com/webmaster/ping.aspx?siteMap=

Here's the class that all my site maps extend from:
public class RenderSitemapXml extends RenderXml {
    public RenderSitemapXml(Document doc) {
        super(doc);
    }

    protected static Document createSiteMapDocument() {
        DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
        DocumentBuilder builder = null;
        try {
            builder = factory.newDocumentBuilder();
        } catch (ParserConfigurationException e) {
            throw new UnexpectedException(e);
        }
        Document doc = builder.newDocument();

        // Create root (urlset) element
        Element root = doc.createElement("urlset");
        root.setAttribute("xmlns", "http://www.sitemaps.org/schemas/sitemap/0.9");
        root.setAttribute("xmlns:geo", "http://www.google.com/geo/schemas/sitemap/1.0");
        root.setAttribute("xmlns:xsi", "http://www.w3.org/2001/XMLSchema-instance");
        String xsiVersion = "http://www.sitemaps.org/schemas/sitemap/0.9";
        String xsiXsd = "http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd";
        root.setAttribute("xsi:schemaLocation", xsiVersion + " " + xsiXsd);
        doc.appendChild(root);

        return doc;
    }

    protected static Element createUrl(Document doc, String loc, String changefreq, Double priority) {
        Element url = doc.createElement("url");
        appendTextElement(url, "loc", loc);
        appendTextElement(url, "changefreq", changefreq);
        appendTextElement(url, "priority", priority);
        return url;
    }

    protected static void appendTextElement(Element parent, String name, Object value) {
        if (value == null) {
            return;
        }

        Document doc = parent.getOwnerDocument();
        Element child = doc.createElement(name);
        parent.appendChild(child);
        child.appendChild(doc.createTextNode(value.toString()));
    }

}


An example Controller:
    public static void siteMap() {
        List<ServiceType> serviceTypes = ServiceType.findAll();
        List<City> cities = City.findAll();
        throw new RenderSitemap(serviceTypes, cities);
    }


An example site map:
public class RenderSitemap extends RenderSitemapXml {
    private final static String[] pages = { "/help", "/providers/login", "/legal",
            "/users/privacy", "/feedback/new", "/serviceTypes/suggest",
            "/providers/forgotPassword", "/support/new" };

    public RenderSitemap(List<ServiceType> serviceTypes, List<City> cities) {
        super(getDocument(serviceTypes, cities));
    }

    private static Document getDocument(List<ServiceType> serviceTypes, List<City> cities) {
        String currentLang = Lang.get();
        // The main language for my site is Spanish
        Lang.set("es");

        Document doc = createSiteMapDocument();
        Element root = doc.getDocumentElement();

        String homeLoc = Router.getFullUrl("ServiceTypes.all");
        root.appendChild(createUrl(doc, homeLoc, "daily", 0.9));

        for (String page : pages) {
            String pageLoc = Request.current().getBase() + page;
            root.appendChild(createUrl(doc, pageLoc, "daily", 0.1));
        }

        ...
        Lang.set(currentLang);
        return doc;
    }

--

[Olivier Refalo]

Will try your code, but you should make a module from it

[Steren]

Hi,
Based on Dirk's code, I started to work on the sitemap generator. The first code of the module is here: https://github.com/Steren/play-sitemap
In addition to Dirk's way to do it, I added an annotation "@InSiteMap()" that allows to add simple actions to the SiteMap.
This code hasn't been tested in production, it's in development.

I need help and have two questions :
* I want to get all actions from all controllers which have this annotation, so I need to get the list of the controllers of the application, how can I get it?

* I'm not a sitemap expert, do we have to push to Search engines the SiteMap XML each time a new version is generated? Or do they regularly check the URL we gave them?

You are also welcomed to criticize the current implementation and propose something better.

Thanks for your help.

Steren
http://www.beansight.com

[Olivier Refalo]

Great! will give it a try later today

[Olivier Refalo]

Let me answer your questions:

* I want to get all actions from all controllers which have this annotation, so I need to get the list of the controllers of the application, how can I get it?

hum.. I would say introspection.

controllers, live in the package app.controllers.*, they also extend import play.mvc.Controller at some point in their hierachy

* I'm not a sitemap expert, do we have to push to Search engines the SiteMap XML each time a new version is generated? Or do they regularly check the URL we gave them?

They come and grab it at regular intervals

Olivier

[Steren]

Thanks,

Let me answer your questions:

* I want to get all actions from all controllers which have this annotation, so I need to get the list of the controllers of the application, how can I get it?

hum.. I would say introspection.

controllers, live in the package app.controllers.*, they also extend import play.mvc.Controller at some point in their hierachy

Does Play! provide a library to easily get all the classes that extend play.mvc.Controller ? Or do you have any suggestion ? I would try to do it using http://code.google.com/p/reflections/.

Thanks again for your support,

Steren

[Olivier Refalo]

I don't know.

But either ways you probably an easier way...

There is no need to index a controller that is not referenced in the route file, right ?

so you can just go route file -> controllers

Play has the api to access the route mappings 

Olivier

[Ricardo Nascimento]

Hello,

You can get all controllers with this line : 

List<Class> classes = Play.classloader.getAssignableClasses(Controller.class);

And if you want get all method with the annotation @InSiteMap : 

List<Method> befores = Java.findAllAnnotatedMethods(Controller.getControllerClass(), InSiteMap.class);

[Antoine Michel]

Does Play! provide a library to easily get all the classes that extend play.mvc.Controller ? Or do you have any suggestion ? I would try to do it using http://code.google.com/p/reflections/.

This should work :

List<Class> controllers = Play.classloader.getAssignableClasses(Controller.class); 

[Steren]

Thanks a lot Antoine and Olivier.
I did a small update to get all annotated actions. The code is still experimental but you can find it here : https://github.com/steren/play-sitemap

We will try it and see how it goes in production.

Steren