Stale Results

55 views
Skip to first unread message

lightyrs

unread,
Oct 2, 2011, 9:09:53 PM10/2/11
to hnse...@googlegroups.com
This might be due to my initial experiments with the api (which didn't include caching and throttling) but as of 4 days ago, all queries to HNSearch api return stale results.

For example:


returns:

{
    "facet_results": {
        "fields": {},
        "queries": {}
    },
    "hits": 58,
    "request": {
        "boosts": {
            "fields": {},
            "filters": {},
            "functions": {}
        },
        "facet": {
            "fields": {},
            "queries": []
        },
        "filter": {
            "fields": {
                "create_ts": [
                    "[NOW-30DAYS TO NOW]"
                ],
                "type": [
                    "submission"
                ]
            },
            "queries": []
        },
        "highlight": {
            "fragments": {
                "include": false,
                "markup_text": false,
                "maxchars": 100
            },
            "include_matches": false,
            "markup_items": false
        },
        "limit": 10,
        "q": "jquery",
        "sortby": "create_ts desc",
        "start": 0,
        "stats": {},
        "weights": {
            "discussion.sigid": 1.0,
            "domain": 1.0,
            "parent_sigid": 1.0,
            "text": 1.0,
            "title": 1.0,
            "type": 1.0,
            "url": 1.0,
            "username": 1.0
        }
    },
    "results": [
        {
            "item": {
                "_id": "3050187-faada",
                "cache_ts": "2011-10-02T09:24:39Z",
                "create_ts": "2011-09-28T22:27:52Z",
                "discussion": null,
                "domain": "blog.jquery.com",
                "id": 3050187,
                "num_comments": 19,
                "parent_id": null,
                "parent_sigid": null,
                "points": 67,
                "text": null,
                "title": "JQuery 1.7 beta 1 released",
                "type": "submission",
                "url": "http://blog.jquery.com/2011/09/28/jquery-1-7-beta-1-released/",
                "username": "petercooper"
            },
            "score": 2.0264915999999999
        },
        {
            "item": {
                "_id": "3050186-bf619",
                "cache_ts": "2011-10-02T09:24:39Z",
                "create_ts": "2011-09-28T22:27:34Z",
                "discussion": null,
                "domain": "addyosmani.com",
                "id": 3050186,
                "num_comments": 0,
                "parent_id": null,
                "parent_sigid": null,
                "points": 3,
                "text": null,
                "title": "JQuery 1.7 Beta Release Preview & Walkthrough",
                "type": "submission",
                "url": "http://addyosmani.com/blog/jquery-17-preview/",
                "username": "petercooper"
            },
            "score": 1.7369927999999999
        },
        {
            "item": {
                "_id": "3049803-2c96a",
                "cache_ts": "2011-10-02T09:24:39Z",
                "create_ts": "2011-09-28T20:39:34Z",
                "discussion": null,
                "domain": "designzzz.com",
                "id": 3049803,
                "num_comments": 0,
                "parent_id": null,
                "parent_sigid": null,
                "points": 2,
                "text": "Coding a Sleek PHP jQuery Contact Form with Runtime Preview with download able source",
                "title": "Coding a Sleek PHP jQuery Contact Form with Runtime Preview",
                "type": "submission",
                "url": "http://www.designzzz.com/coding-php-jquery-contact-form-script-runtime-preview/",
                "username": "wickedsunny1"
            },
            "score": 1.4555910000000001
        },
        {
            "item": {
                "_id": "3047860-b5521",
                "cache_ts": "2011-10-02T09:14:15Z",
                "create_ts": "2011-09-28T14:03:01Z",
                "discussion": null,
                "domain": "spin.atomicobject.com",
                "id": 3047860,
                "num_comments": 0,
                "parent_id": null,
                "parent_sigid": null,
                "points": 2,
                "text": null,
                "title": "Optimize page speed with jQuery waypoints",
                "type": "submission",
                "url": "http://spin.atomicobject.com/2011/09/23/optimize-page-speed-with-jquery-waypoints/",
                "username": "atomic_object"
            },
            "score": 2.0264915999999999
        },
        {
            "item": {
                "_id": "3046908-723e1",
                "cache_ts": "2011-10-02T09:09:02Z",
                "create_ts": "2011-09-28T07:43:47Z",
                "discussion": null,
                "domain": "techblaster.net",
                "id": 3046908,
                "num_comments": 0,
                "parent_id": null,
                "parent_sigid": null,
                "points": 1,
                "text": "This is one of the easiest method to add a Jquery slideshow to a blogger blog. You can add it inside a post, just above the post, side bar of the blog, at the footer of the blog or anywhere else easily. Read this and learn how to do it in step by step manner.",
                "title": "Add Slideshow to a blogger blog (Jquery)",
                "type": "submission",
                "url": "http://www.techblaster.net/2011/09/how-to-add-slideshow-to-blog-jquery.html",
                "username": "Techmaker"
            },
            "score": 2.0315523
        },
        {
            "item": {
                "_id": "3046618-32b27",
                "cache_ts": "2011-10-02T09:03:51Z",
                "create_ts": "2011-09-28T05:33:02Z",
                "discussion": null,
                "domain": null,
                "id": 3046618,
                "num_comments": 1,
                "parent_id": null,
                "parent_sigid": null,
                "points": 2,
                "text": "I want to build dynamic web apps.<p>So far I have gone through web tutorials (w3 schools) of HTML, CSS, jQuery and score about 75% at the end of each tutorial. I'm now going to go through AJAX and SQL tutorials on the same site. Next I'm going to try Michael Hartl's Rails tutorial.<p>Say I become 65 or 70% competient with the above technologies (while being slow to build), am I ready to build dynamic web apps, or should I add something to my to-learn list? Are there any holes or gaps in my plan? I also want my apps to look nice and feel fast.<p>I used to know C++ fairly well about 5 years ago. No coding ever since.<p>To get an idea what I'm trying to build think something like a simple groupon-esq featured online college application tool (so you can apply for multiple colleges/scholarships/funds at the same time).a",
                "title": "Ask HN: What web technologies should a web virgin learn?",
                "type": "submission",
                "url": null,
                "username": "no-go-mojo"
            },
            "score": 0.25303003000000002
        },
        {
            "item": {
                "_id": "3044339-a82cd",
                "cache_ts": "2011-10-02T08:53:27Z",
                "create_ts": "2011-09-27T17:35:54Z",
                "discussion": null,
                "domain": "firstamong.com",
                "id": 3044339,
                "num_comments": 0,
                "parent_id": null,
                "parent_sigid": null,
                "points": 1,
                "text": null,
                "title": "Most Powerfull jQuery Feed Plugin",
                "type": "submission",
                "url": "http://www.firstamong.com/nannufeed",
                "username": "neinsun"
            },
            "score": 2.0264915999999999
        },
        {
            "item": {
                "_id": "3043846-43d96",
                "cache_ts": "2011-10-02T08:53:27Z",
                "create_ts": "2011-09-27T15:59:25Z",
                "discussion": null,
                "domain": "gauth.fr",
                "id": 3043846,
                "num_comments": 1,
                "parent_id": null,
                "parent_sigid": null,
                "points": 5,
                "text": null,
                "title": "How to spider a HTML page with jQuery",
                "type": "submission",
                "url": "http://gauth.fr/2011/09/how-to-spider-a-html-page-with-jquery/",
                "username": "Gauth"
            },
            "score": 2.0264915999999999
        },
        {
            "item": {
                "_id": "3043345-f86e6",
                "cache_ts": "2011-10-02T08:48:14Z",
                "create_ts": "2011-09-27T14:17:53Z",
                "discussion": null,
                "domain": "readwriteweb.com",
                "id": 3043345,
                "num_comments": 0,
                "parent_id": null,
                "parent_sigid": null,
                "points": 1,
                "text": null,
                "title": "Microsoft Makes the Case for More jQuery, Fewer Dependencies",
                "type": "submission",
                "url": "http://www.readwriteweb.com/hack/2011/09/microsoft-makes-the-case-for-m.php",
                "username": "FrancescoRizzi"
            },
            "score": 1.7369927999999999
        },
        {
            "item": {
                "_id": "3042844-4a7b1",
                "cache_ts": "2011-10-02T08:48:14Z",
                "create_ts": "2011-09-27T11:48:20Z",
                "discussion": null,
                "domain": "netmagazine.com",
                "id": 3042844,
                "num_comments": 0,
                "parent_id": null,
                "parent_sigid": null,
                "points": 2,
                "text": null,
                "title": "Create an interactive street view with jQuery",
                "type": "submission",
                "url": "http://www.netmagazine.com/tutorials/create-interactive-street-view-jquery",
                "username": "ahdanielsan"
            },
            "score": 2.0264915999999999
        }
    ],
    "time": 0.034785985946655273,
    "warnings": []
}

Please help me resolve this.

Thanks,

Harris

Andres Morey

unread,
Oct 3, 2011, 12:30:20 AM10/3/11
to hnse...@googlegroups.com
Hi Harris,

Thanks for letting us know about the problem. It looks like our indexing script went down over the weekend but it should be working again:


We're working on some improvements that should make everything more robust.

Andres

Harris Novick

unread,
Oct 3, 2011, 10:54:23 AM10/3/11
to hnse...@googlegroups.com
Thank you for the quick turnaround and for being so responsive.

Everything seems to be working now.

Best,

Harris

Sent from my iPhone

Harris Novick

unread,
Oct 12, 2011, 6:43:14 PM10/12/11
to hnse...@googlegroups.com
Hey Andres,

Is it just me or is the api down?

Thanks,

Harris

On Sun, Oct 2, 2011 at 9:09 PM, lightyrs <harrisda...@gmail.com> wrote:

Andres Morey

unread,
Oct 12, 2011, 7:11:51 PM10/12/11
to hnse...@googlegroups.com
Hi Harris,

The API is working but results are stale as of midnight last night. We're upgrading the search engine so index updates are off right now. Everything should be up-to-date by tomorrow.

Andres

Harris Novick

unread,
Oct 12, 2011, 7:44:51 PM10/12/11
to hnse...@googlegroups.com
Thanks Andres.

Harris Novick

unread,
Oct 18, 2011, 10:05:21 PM10/18/11
to hnse...@googlegroups.com
Hey,

Is the API down again?

Sent from my iPad

Andres Morey

unread,
Oct 19, 2011, 1:48:59 AM10/19/11
to hnse...@googlegroups.com
Everything should be up-to-date now.

Andres

Jared Flatow

unread,
Oct 19, 2011, 1:51:11 AM10/19/11
to hnse...@googlegroups.com
Did the API somehow change? My hotness results have been broken for a few days:


Thanks,
jared

Andres Morey

unread,
Oct 19, 2011, 2:21:18 AM10/19/11
to hnse...@googlegroups.com
We upgraded to a new solr binary which has stricter validation rules. To fix the problem with the hotness results in popdots, reverse 'NOW' and 'create_ts' in the ms() function.

Andres
Reply all
Reply to author
Forward
0 new messages