Jira (PDB-112) Provide means to request events from latest report in timeframe

2 views
Skip to first unread message

Kenneth Barber (JIRA)

unread,
Jan 12, 2015, 9:30:29 AM1/12/15
to puppe...@googlegroups.com
Kenneth Barber updated an issue
 
PuppetDB / New Feature PDB-112
Provide means to request events from latest report in timeframe
Change By: Kenneth Barber
Story Points: 3
Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v6.3.10#6340-sha1:7ea293a)
Atlassian logo

Beth Cornils (JIRA)

unread,
Jul 14, 2015, 12:16:20 PM7/14/15
to puppe...@googlegroups.com
Beth Cornils commented on New Feature PDB-112
 
Re: Provide means to request events from latest report in timeframe

Ryan Senior what's old is new again. It's be great if we could assess when we can get this in. I'd like to make some changes to EI and it appears this is a blocker.

Let's touch base when we know what we can do in Ankeny between HA and Unchanged.

This message was sent by Atlassian JIRA (v6.4.5#64020-sha1:78acd6c)
Atlassian logo

Wyatt Alt (JIRA)

unread,
Jul 19, 2015, 5:18:13 PM7/19/15
to puppe...@googlegroups.com
Wyatt Alt commented on New Feature PDB-112

Reid Vandewiele Deepak Giridharagopal Kenneth Barber Ryan Senior how important is the ability to specify a timeframe with an endpoint other than current time? This becomes simpler if we're concerned only about events from the latest reports for all nodes, rather than the events from the latest reports prior to an arbitrary date. We already hold the ids for the latest reports in the certnames table, and have a report_id column in resource_events, which makes the selection of "all events from latest reports for all nodes" pretty quick. If we don't care about the arbitrary end date, I think this would be a lot more straightforward and performant.

Wyatt Alt (JIRA)

unread,
Jul 19, 2015, 5:30:13 PM7/19/15
to puppe...@googlegroups.com
Wyatt Alt commented on New Feature PDB-112

that's kind of the whole point of this I suppose. Just trying to think of ways we could do this without hitting the performance issues we currently see with distinct_resources.

Scott Walker (JIRA)

unread,
Jul 20, 2015, 6:28:12 AM7/20/15
to puppe...@googlegroups.com
Scott Walker commented on New Feature PDB-112

Joseph Wagner Beth Cornils Verne Lindner Wyatt Alt Pinging half the world here just to make sure our expectations are aligned. This ticket directly affects how "Events" works in config management.

I'd figured that filtering by latest report gives us better usability for the reasons discussed about and as a side effect reduces the usefulness of filtering by time. It makes sense to do both: first filter to get only events from a latest report, then filter by a time period. That works a lot better for a 5 min interval than a really long one. You'd also expect to see fairly minimal differences between 1 hour and 12 hours in most systems though, as you'd expect nearly all nodes to have reported within the last hour.

Beth Cornils (JIRA)

unread,
Jul 20, 2015, 1:34:25 PM7/20/15
to puppe...@googlegroups.com
Beth Cornils commented on New Feature PDB-112

Hi Scott Walker and Deepak Giridharagopal

In talking with Jeremy Adams and Chris Barker I think the way we handle EI is correct. The reason the counts for SO and SI do not match is because we had an issue where failed events during a 24 hour time period were obfuscated. This was viewed as a miss.

Given that, I think making this requested change as an additional option might make more sense. I'm not sure we want to remove visibility to the events that happen in the time frame selected.

Verne Lindner

Reid Vandewiele (JIRA)

unread,
Jul 20, 2015, 3:08:14 PM7/20/15
to puppe...@googlegroups.com

Wyatt Alt Scott Walker as Wyatt said, the ability to specify an endpoint other than the current time is kind of the point.

The specific Event Inspector (EI) desired use case that originally drove this ticket was trying to achieve a kind of freeze-frame picture of events from the latest report.

The problem with the existing ["=", "latest-report?", true] query is that if in EI users pull up a view of events from classes based on that query, spend two or three minutes looking at it and then try to drill in to a specific class, it's possible that relevant nodes will have submitted an updated report in the intervening time and there are no longer any events for that class that match ["=", "latest-report?", true].

The desired user experience is that when users first load EI they are presented with a view of the world as it looked at the time of page load. They should be able to muse over it, navigate around inside it, and expect consistency inside of it, at least until they refresh the browser.

Right now because EI makes multiple queries back to PuppetDB as the user is navigating around inside of it, to present that consistent freeze-frame of "current" events it seems like we would need some kind of ["=", "latest-report-before", "2013-10-30T14:55:00-07:00"]'.

That's the use case that originally drove the ask in the ticket.

Joseph Wagner (JIRA)

unread,
Jul 20, 2015, 3:25:13 PM7/20/15
to puppe...@googlegroups.com
Joseph Wagner commented on New Feature PDB-112

+1 to everything Reid Vandewiele states. It is ideal to look at the latest run reports when State Inspector loads, but the endpoint needs to remain the time of loading as users browse the interface, otherwise there will be confusing instances where the ground shifts from under them while they're trying to investigate something because Puppet has run again and the data changes. Do we currently query PuppetDB every time a user navigates between panes in State Inspector or do we just query when a user first arrives at State Inspector?

Beth Cornils (JIRA)

unread,
Jul 20, 2015, 4:52:11 PM7/20/15
to puppe...@googlegroups.com
Beth Cornils commented on New Feature PDB-112

Joseph Wagner and Reid Vandewiele I think for the moment my concern is, we'd change the user experience from what they have today. While the API end point could remain for the current configuration, the UI would need adjustments to allow both views. 2 days doesn't feel like enough time to meet that objective safely.

With that said, are we comfortable with making this noticeable shift in Ankeny? I think that seems fine. I could be missing something though.

So confirm if you think we are comfortable leaving things as they are for SG.

Wyatt Alt, Deepak Giridharagopal

Joseph Wagner (JIRA)

unread,
Jul 20, 2015, 5:02:13 PM7/20/15
to puppe...@googlegroups.com
Joseph Wagner commented on New Feature PDB-112

My understanding (please correct me anyone if I'm wrong) is If we leave things as they are, we're shipping a brutal performance at large scale. This isn't a regression, we apparently already had these performance issues, but in SG we're explicitly optimizing for a larger scale so this is a much greater concern now. Im fine with the functionality not changing for SG, but doesn't that mean it is unusable at target scale?

Reid Vandewiele (JIRA)

unread,
Jul 20, 2015, 5:57:13 PM7/20/15
to puppe...@googlegroups.com

I'll volunteer as a point of interest that if necessary for performance reasons, I wouldn't be overly opposed to a latest-report-based view that did not necessarily have the idealized consistency characteristics. That is, I'd more or less accept an "events right now" view wherein events could possibly disappear, so long as I could switch back to e.g. the 5-minute or 30-minute view if I needed to to keep pursuing a specific event. I think latest-report-before is ideal, but as a practitioner I'd still be thumbs-up on using latest-report as a useful MVP.

I 100% defer to Joe with regards to whether or not that would be acceptable product-level UX though, or if shipping that kind of thing would prevent us from achieving the ideal later.

Wyatt Alt (JIRA)

unread,
Jul 20, 2015, 10:11:04 PM7/20/15
to puppe...@googlegroups.com
Wyatt Alt commented on New Feature PDB-112

To give an update on this, for the moment we've gone ahead with retaining present behavior for aggregate-event-counts, rather than shifting to a latest-report based view. In light of that, there's no immediate performance-based reason to limit this to latest report only. Since the pressing perf question is settled, I don't see a reason not to target the full functionality described in the ticket.

Verne Lindner (JIRA)

unread,
Apr 8, 2016, 7:41:03 PM4/8/16
to puppe...@googlegroups.com
Verne Lindner commented on New Feature PDB-112

Wyatt Alt Karel Brezina So many comments on this ticket, that I need to ask: can we do this in time for us to fix PE-1906 for Couch?
Beth Cornils Steve Quin fyi

This message was sent by Atlassian JIRA (v6.4.13#64028-sha1:b7939e9)
Atlassian logo

Wyatt Alt (JIRA)

unread,
Apr 8, 2016, 7:50:03 PM4/8/16
to puppe...@googlegroups.com
Wyatt Alt commented on New Feature PDB-112

Verne Lindner per the discussion in the email thread you sent on 3/9, my understanding is that we're shelving this in favor of the more limited functionality proposed in that email/enabled by PDB-2527. If that's the case, I don't think Couch timeframe is an issue. I'll pull that ticket in as ready for engineering, but am I misunderstanding the game plan?

Verne Lindner (JIRA)

unread,
Apr 11, 2016, 1:24:04 PM4/11/16
to puppe...@googlegroups.com
Verne Lindner commented on New Feature PDB-112

Wyatt Alt Nope - that's it. Do you have a rough idea of when you'll be done, so we can plan for our team's work?
Filip Hrbek fyi

Wyatt Alt (JIRA)

unread,
Apr 11, 2016, 4:26:50 PM4/11/16
to puppe...@googlegroups.com
Wyatt Alt commented on New Feature PDB-112

Verne Lindner I stuck it in our hopper, which means its on deck. The team is tied up all this week, but I'm happy to personally take it up next.

Verne Lindner (JIRA)

unread,
Apr 11, 2016, 5:50:04 PM4/11/16
to puppe...@googlegroups.com

Russell Mull (JIRA)

unread,
May 17, 2017, 12:51:03 PM5/17/17
to puppe...@googlegroups.com
Russell Mull updated an issue
 
Change By: Russell Mull
Labels: triaged
This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe)
Atlassian logo

Moses Mendoza (JIRA)

unread,
May 18, 2017, 1:49:10 PM5/18/17
to puppe...@googlegroups.com

Claudia Petty (Jira)

unread,
Jun 21, 2023, 10:56:10 AM6/21/23
to puppe...@googlegroups.com
Claudia Petty updated an issue
Change By: Claudia Petty
Labels: new-feature
This message was sent by Atlassian Jira (v8.20.21#820021-sha1:38274c8)
Atlassian logo
Reply all
Reply to author
Forward
0 new messages