Timestamped events

[Thom DeCarlo]

Hi,

I've got a task where I need to create a simulation that loads a file that contains timestamped data (like the attached .txt file). Then it will create SimEntities when the simulation time passes the timestamp for each data record. The SimEntity will store the data internally and carry it through the rest of the simulation, eventually printing the data or writing it to a log file.

I thought I could do this via a variation of the FileToMatrix object. I created a new function, based on that object called "FileToStringMatrix". The main difference is that I've replaced the "double[][]" matrices with "String[][]".  Unfortunately, I'm not getting any values from that and the output panel shows "Value     [Ljava.lang.String;@3a9bc629". That looks like the address of the array instead of the value in the array.

I also looked at loading the data as a TimeSeries (via the attached .inc file). But there are two problems with that. The TimeSeries Value operation doesn't seem to know what to do with the free-text field and the entities are being generated at regular intervals instead of at the event timestamp. 

I'm guessing that the second problem might be simply because I'm using the TimeSeries incorrectly. But, I don't know if the first one can be fixed without code changes.

What do you think?

-Thom

[Thom DeCarlo]

An update...

I discovered that, even though the value looks funky on the FileToStringMatrix OutputViewer panel, I am able to assign the correct values out of that object into my SimEntities. Yay!

So, my SimEntities now contain two String attributes, timestamp and databundle. The next trick is to somehow create a server that delays ins inbound SimEntities until the simulation time is greater than the value of the entity's timestamp value. Something like that is possible, right? (fingers crossed)

-Thom

[Mark]

EDIT:

Can you convert your string timestamp to simulation time (double or float) or import the equivalent in the same way you currently import the string?

Then it would just be as simple as delaying the entity for an additional (TimeStampDecimal - this.obj.SimTime).

NOTE:  I may have the wrong syntax for current simulation time but it's easy to lookup in the documentation.

[Harry King]

Hi Thom,

 

I discovered that, even though the value looks funky on the FileToStringMatrix OutputViewer panel, I am able to assign the correct values out of that object into my SimEntities. Yay!

HK - Yes, that is correct. The Output Viewer just doesn't know what to do with an output that returns a String[][], but the data is available to the expression system.

 

So, my SimEntities now contain two String attributes, timestamp and databundle.

HK - It would be possible to provide a function that would convert your time stamp string to a number. I'm not sure if I like that solution though. It might be possible to generalise FileToVector and FileToMatrix to handle non-numerical data. The most straightforward approach would be to modify your FileToString object to work with your specific data set and return two separate outputs: timestamp would be a double[] of simulation times, and databundle would be the matching String[] of text data.

 

The next trick is to somehow create a server that delays ins inbound SimEntities until the simulation time is greater than the value of the entity's timestamp value. Something like that is possible, right? (fingers crossed)

HK - Assuming that you have a number representing the timestamp, you can use an EntityDelay object whose Duration input is set to 'this.obj.timestamp - this.SimTime'. 

I also looked at loading the data as a TimeSeries (via the attached .inc file). But there are two problems with that. The TimeSeries Value operation doesn't seem to know what to do with the free-text field and the entities are being generated at regular intervals instead of at the event timestamp. 

HK - The TimeSeries object expects to receive a number for each time stamp. It doesn't know what to do with a string.

 

Harry

[Harry King]

Hi Thom,

We are planning to add a function to the expression system that will convert a string to a number. It will appear in the next release (2017-08).

Harry

[Thom DeCarlo]

Hi Harry, 

If I'm interpreting your statement properly, I don't think this gives quite the capability that I'm looking for. I think you are saying that the expression system will support changing numbers that are provided as Strings into actual numbers (Integer, Float, Double, etc) objects for use in performing math operations. I'm looking for something that just carries the String along as a String data object within the SimEntity' attribute list. 

I have created a fork off of the main JaamSim GitHub project where I'm working in that direction. (https://github.com/pixelpshr/jaamsim, timestamped-record branch) In there, I've created a few new objects. Initially, I created the FileToStringVector and FileToStringMatrix objects, based on your guidance provided earlier in this message thread. These now appear in the BasicObjects palette. 

I'm currently working on a new objects, datatype/TSR.java and datatype/TRSList.java. TSR.java is the TimeStampedRecord object, which contains a javafx.util.Pair.Pair<ZonedTimeStamp, String> private member. The TSRList.java is, obviously, a List of those objects. 

I think I've got most of the code working to read a time-stamped record text file into the TSRList, but I'm still having trouble extracting the Value of the list elements. I expect to be working on this in the coming week. 

My ultimate goal is to be able to read a list of timestamped records, either at the start of the simulation, or when the "FileToTSRList" object is encountered by a SimEntity. The TSRList would then be used to generate new SimEntities when the Simulation Time passes the timestamp for the record, probably with a variant of the System or EntityDelay object. The new SimEntity would hold a copy of the TSR as an attribute and, minimally, print it to an output file when the EntityLogger object is encountered. In the longer term, the SimEntity would take different branches within the simulation, depending on the contents of the TSR.

I hope that clears up what I'm trying to do. Maybe you can see a better way to pull this off!

Thanks,

Thom

[Harry King]

Hi Thom,

My idea was to use your FileToStringMatrix object to read and store the data as a matrix of strings. Then, each entity could index into the appropriate row of the matrix and use an Assign object to convert the first entry from a time stamp string to a number in seconds, which would then be stored as an attribute of entity. The second entry would be just assigned as a string to another attribute of the entity. The input to the AttributeAssignmentList input would be something like:

{'this.obj.timestamp = timeValue( FileToStringMatrix1(this.obj.n)(1) )} {'this.obj.databundle = FileToStringMatrix1(this.obj.n)(2)'}

where the entity's attribute n is the appropriate row in the matrix and the new function timeValue would convert a timestamp string to a number in seconds.

Would that approach work for you model?

Harry

[Thom DeCarlo]

Hi Harry,

I think that will work. It does lose the explicit connection between the timestamp and its databundle, but I can deal with that in the code. Where would the timeValue() function reside? And, can it be smart enough account for timestamp in milliseconds, as well as seconds? Finally, how would I compare the stored timestamp with the simulation time? I'd need that to introduce a timestamp-defined delay into the System+Queue elements.

Thanks for thinking about this! It's helping a lot.

-Thom

[Harry King]

Hi Thom,

 

I think that will work. It does lose the explicit connection between the timestamp and its databundle, but I can deal with that in the code.

HK - They would be connected though the entity -- each entity would have the timestamp and databundle for a single record.

 

Where would the timeValue() function reside?

HK - The timeValue function has not be added yet. We will get it into the next release. 

And, can it be smart enough account for timestamp in milliseconds, as well as seconds?

HK - We will have two separate functions 'timeValue' and 'numberValue' (names not final yet). The first will convert a timestamp string into a number with the unit of time, while the second will convert a number string into a dimensionless number. The user will be responsible for applying the appropriate unit by multiplying the number by 1.0 with the desired unit, e.g. 'numberValue("100") * 1[ms]' converts the string "100" into the value 100 milliseconds.

 

Finally, how would I compare the stored timestamp with the simulation time? I'd need that to introduce a timestamp-defined delay into the System+Queue elements.

HK - Once you have a number with unit of time, say this.timestamp, the delay to reach that simulation time is just 'this.timestamp - this.SimTime'. Note that this expression assumes that an input of 0 milliseconds corresponds to zero simulation time. If not, you will need to subtract a suitable offset from the equation.

 

Harry

[Harry King]

Hi Thom,

We have taken a different approach than the one I described in my previous post. Instead of reading the data as an array of strings, we have modified the original FileToVector and FileToMatrix objects to read records containing any type of input that can be expressed by an expression: numbers with or without units, strings, entities, arrays, etc. We have also included an new keyword DataFormat that so that you can enter a string as abc or 'a b c' instead of '"abc"' or '"a b c"'. it also allows you to identify an input as a timestamp.

If you would like to test this yourself, you can pull the HarryWIP branch from GitHub. The keyword pop-up for DataFormat should give you all the information you need to use this feature.

Harry

[Harry King]

Hi Thom,

The improved FileToVector and FileToMatrix objects are included in release 2017-08, and the new features are described in the User Manual. Also see the two examples that I posted today.

Harry

[Thom DeCarlo]

Harry,
Thanks for the update! I will spend a lot of time working on this effort this week. I will definitely pull down your branch for testing.

Thanks again!
Thom

--
You received this message because you are subscribed to a topic in the Google Groups "Jaamsim Users Discussion Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jaamsim-users/gyFx286OceQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jaamsim-user...@googlegroups.com.
Visit this group at https://groups.google.com/group/jaamsim-users.
To view this discussion on the web, visit https://groups.google.com/d/msgid/jaamsim-users/ba5a381a-eb37-412a-9ed0-9610a1fe599f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Thom DeCarlo]

Harry,

I'm trying to figure out how the time unit attributes are displayed in the Text graphic object. 

The AttributeDefinition for the variable that the timestamp gets read into is "{  timestamp  0  [h]  }". I'm not really certain what should be inside the square braces.

The Text object "Format" keyword is 'timestamp = %.3f'

When the sim runs, I am shown a very large floating point number. The time that was read from the file is "2017-06-01T12:09:00.000", but the floating point number is "63621202140.000". Converting that timestamp string into epoch seconds, it should be "1496318940.000" 

So, I'm not sure what the 636... number represents.

Also, is there a way to display the time value as a string, as in the format of the string that was read from the file?

Thanks!

Thom

[Thom DeCarlo]

For clarification, when I ask about displaying the time value as a string, I mean to display it in the same format as the OverlayClock object.

Thanks again!

Thom

[Harry King]

Hi Thom,

 

The AttributeDefinition for the variable that the timestamp gets read into is "{  timestamp  0  [h]  }". I'm not really certain what should be inside the square braces.

HK - This input is correct, but the preferred way to enter it is { timestamp 0[h] }. It defines the attribute and sets its initial value to 0 hours.

 

The Text object "Format" keyword is 'timestamp = %.3f'

HK - Unfortunately, there is no easy way to display timestamp value in the timestamp format that was read from the file once it has been converted into seconds. You could save the value as a string before converting it to seconds, but you would need a function to do this. Or, we could provide a function that converts a number with units of time into a timestamp string.

 

When the sim runs, I am shown a very large floating point number. The time that was read from the file is "2017-06-01T12:09:00.000", but the floating point number is "63621202140.000". Converting that timestamp string into epoch seconds, it should be "1496318940.000" 

So, I'm not sure what the 636... number represents.

HK - JaamSim works with a simplified calendar that does not include leap years and leap seconds. The timestamp 0000-01-01T00:00:00.000 is zero seconds in simulation time.

 

Also, is there a way to display the time value as a string, as in the format of the string that was read from the file?

HK - See my second response above.

Harry

[Harry King]

Thom,

Further to JaamSim's use of a simplified calendar, there is no real reason why it couldn't just use the calendar functions provided by Java. The main users of the timestamp format are the OverlayClock and TimeSeries objects. The OverlayClock is used only as a display and TimeSeries is normally used to interpret historical data which DOES have leap years and leap seconds. Deleting the entries for Feb. 29 in a historical data set is always an awkward extra step that could be made unnecessary. The only motivation for using the simplified calendar is to avoid the confusion of starting having a multi-year simulation run end before Dec. 31 when it started on Jan. 1.

What are your thoughts on this question?

Harry

[Thom DeCarlo]

Hi Harry,

I would hate to be the cause of a major rewrite of your software just to satisfy my seemingly unusual need. Switching from the simple calendar to Java Time features seems likely to cause problems for existing simulations and confusion for other JaamSim users. For example, not only does the Java Time account for leap year and leap seconds, it also has local time vs time zone offsets, and varying resolution timers and intervals. 

In the case of my simulation, I need to read timestamped data (that now works. Thanks!) and use those timestamps to delay the processing of actions on entities that carry the data. Think of it like a queuing system at a restaurant. A person makes a dinner reservation for a party of four. The restaurant cannot provide a spot in the queue for the requested time, so they negotiate an alternative time. At the specified time the dinner party is seated unless there are missing party members. In that case, the group is placed back in the queue where they must wait for the missing members. When all members are present, they get priority for seating, but cannot be guaranteed the next available table since it may be designated for another group's reservation. 

The key element here is the restaurant's reservation queue, which holds the diners until a specified time, and the hostess who acts as a gate. Diners (SimEntities) cannot pass through the gate until their time has arrived (SimTime >= Reservation timestamp).

So, perhaps you can suggest a work-around that will let me create such a queue and gate for controlling the flow through my simulation without rewriting your calendar system.

Thanks again for all your help on this!

Thom

[Harry King]

Hi Thom,

Actually, it would not be very difficult to make the change to the Java calendar system. We would assume UTC for all time calculations. However, it sounds like this is not an essential feature for your model, so let's leave this idea to brew for a while.

I assume that the timestamps in the data file are the reservation times for each customer, and that the customers can arrive earlier than their reservations. In that case, you could sent the arriving customers to a Queue that is sorted in the order of the customers' reservation times. To accomplish this, set the Queue's Priority input to 'floor(this.obj.timestamp/15[min])'. The input is divided by 15 minutes to make the value dimensionless and to break the reservation times into 15 minute slots -- the Priority input value must be a dimensionless integer.

Customers that arrive without a reservation are a bit tricky. One design would be to assign them a reservation for the next available slot in the schedule and place them in the same queue as the others. Customers whose fully party has not arrived at the reservation time could be handled in the same way. This scheduling operation would be non-trivial, but could be performed in an Assign object using higher-order functions.

The Queue could be processed by a Seize object that takes the first customer whenever a table becomes available. If the customer's party has not arrived yet, the table would be released by a Release object and the customer returned to the Queue with a new reservation time.

Another design would be to maintain a separate Queues for customers without reservations and for customers whose full party did not arrive on time. These Queues would have separate Seize objects to seize their tables. The order in which separate Seize objects seize a common resource is determined by the Priority value for the first customers in their Queues. In this case, you would assign a sufficiently large Priority value to the two extra Queues so that they are not used unless the main Queue is empty.

Is any of the above what you had in mind?

Harry