Sort sources

[theboo...@gmail.com]

Simon's Modified Default Format had a most useful facility to sort sources by date order.

The little source icon to the left of the list of sources in the event details window did that.

In the default format, it merely duplicates the same icon a little further down in the Attach section.

Any chance of changing it?

Mick

[John Nairn]

I’ll look into that, but it is probably better to add “Sources” to the “Tree->Sort” submenu than try to do within the format (and the icon can link to that menu command too).

The curious thing is that the GEDCOM standard does not have a DATE tag for a source. Since my day job is writting scientific publications where all reference must have dates, I thought that was strange. My solution was to add a custom _DATE tag for SOUR records. A command to sort them can only look for this custom tag (those without a date would sort to the bottom).

Regards,

John

<SortSources.jpeg>

--
You received this message because you are subscribed to the Google Groups "GEDitCOM II Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to geditcom-ii-discu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/geditcom-ii-discussions/ce871b84-e55d-4dee-a7f9-51e40aabd805n%40googlegroups.com.
<SortSources.jpeg>

[Richard Blake]

I've always found the way that GEDCOM dates sources a bit strange. there are two places a date can be entered.

1. In the SOUR record under DATA.EVEN.DATE. This is a DATE_PERIOD and is meant to cover the period that the information in the source covers.

2. In a source citation under DATA.DATE which I believe is designed to be the date on which the information was entered into the source but have heard that some people use this to record the date that they viewed the source.

In the Modified Default Format I believe that the latter is used to sort sources but I guess it depends on what you are trying to achieve.

Regards, Richard

[theboo...@gmail.com]

Thanks, John. I think Simon's sort uses the DATE tag in the source details. This means that if, say, a book was publiched in 1959, I'd add that to the _DATE tage but I'd also have to add it to the date in the source details where I mentioned the page number or whatever, otherwise it wouldn't sort

If the source is something like an English Baptism Register then it might have been used for decades, or even centuries. So the _DATE tag might have something like FROM 1813 TO 1921 whereas the source details DATE tag might have the actual date of the baptism, say 21 SEP 1846.

Of course, I might be doing it all wrong.

Best

Mick

[Simon Robbins]

I must have missed this one, but yes you are right Mick, the sort sources script sorts by the date entered in the citation not the source record. As you say a source may cover a period of time, in some case a very long one, but when I am just picking out one entry I would use the date of that entry in the citation. That may or may not be the same as the event I am using the source to evidence of course. Obviously a baptism is usually entered into the register on the same day as the baptism takes place, but I have seen many which refer to the birth date as well which could be several years earlier (especially for non conformists) or some which have referred to the fact that the father was already deceased at the time of baptism etc. So the event being evidenced has the date of the event, the source citation has the date of the entry concerned and the source record has the period covered by the whole source.

Thinking about what I had done before reminded me there another script I wrote embedded in the format. I have used it quite a few times in the past and found it useful, not sure about others. If you have an individual with a number of events but not a birth date, there is a butt to run a script to calculate an approximate date of birth and enter it in the birth field. If there are a number of events which point to different answers I think it comes up with an average but I think I made it clever enough to spot if the average is later than the earliest event and adjust it's calculation appropriately. If an individual already has a calculated estimate the button will show "recalculate" and when you add some more events it can be refined if necessay.

Simon

[John Nairn]

Thanks Simon.

The new sort source option was inspired by your feature in the Modified Default Format. I think I posted before, but the sorting looks for dates in 3 places:

1. SOUR.DATA.DATE entered in the citation

2. _DATE tag in the source record

3. First data in a DATE.EVEN.DATE tag in the source record

It uses the first one it finds. #1 and #3 are standard GEDCOM tags (but it my opinion an odd way to date a source). #2 is custom one I added for GEDitCOM II to make dates easier to see in the record. The others are allowed too. The problem is #1 would have to be in every citation to that source while #2 and #3 are in the source itself.

I will look for the script you mentioned. I could be an extension (instead of embedded) and they would work for all formats (and any format could add a button to run the extension too).

Did you try the updated “View GEDCOM Source” (option-command-E) that now shows the GEDCOM source editor in the index window, selects the text you are editing, and scrolls it into view? This is alternative to your option to show the relevant text in a separate window, which I think needede Pashua. 

John

To view this discussion on the web visit https://groups.google.com/d/msgid/geditcom-ii-discussions/e21645da-fa09-4dcd-aaed-5e2587fb362an%40googlegroups.com.

[theboo...@gmail.com]

Thanks, John and Simon.

I've used the updated 'View GEDCOM source' several times. Works a treat. Very useful.

Mick

[Simon Robbins]

Thanks John,

I did look at the source sorting. I didn't realise it was there at first, is there actually a menu command for it? I couldn't see one, at least not where I expected to see it in the Tree:Sort submenu. Obviously a much better solution than my Applescript, but obviously one I couldn't implement myself. I'm not quite sure why you think the standard GEDCOM dating of sources is odd though. I agree for something like a census when the date will be the same for the whole country. But for something like baptisms/marriages/burials, I have known churches in small villages who have been using the same registers for well over a hundred years. So the source record might have the name XYZ Church Baptism Register Dated 1815-1921. But the citation for a particular individual might say something like Page 43 Row 6 Date 3/4/1843. I'm not sure why you would need two dates in the source record. Do you create a separate source record for every baptism? I could see a reason for maybe one record per church register but not for every baptism. Personally I just have one source called Baptisms and put the church, page and date details in the citation.

As for the updated View GEDCOM source command it's much easier now that it highlights the correct text and scrolls to the right place. The only observation I would make however is that as soon as you start editing any of the text the highlighting naturally disappears. It would be helpful if there could be some way of making the highlighting persistent whilst editing. 

Is there a reason why you chose not to do it the same way as the original GEDitCOM application. I don't mean GEDitCOM II 1.0, I mean the really old one back before Mac OSX in the late 90s? I think. That didn't use a separate window, as I had to do using Pashua, which I admit was a bit clunky, but was the only way I could do it. Nor did it use the index window and show the whole record GEDCOM which can be a bit daunting for those who are new to viewing raw data. What it was use the same window that is displaying the event/source/name details or whatever and simply switched of the formatting for that window, replacing the window contents with the GEDCOM for that window. I think the view was toggled with Cmd-E so after editing, pressing the same keys again simply re-formatted the window. No need for extra windows, or switching to the index window. It also meant that however much you edited the GEDCOM it didn't matter about highlighting text as it was only the text for that window available to edit and you couldn't stray off to other bits of the record accidentally.

Simon

[theboo...@gmail.com]

There are lots of ways of handling sources and citations. Mine is pretty straightforward and based on my days as an academic historian going back to the mid-1970s. I define a source, and then I cite it in one or more instances. And I agree with John that GEDCOM is a bit odd when it comes to source dates. I'd add that it's a bit odd, all round, when it comes to sources.

For me, a source is a document/artifact/photo ... whatever.. It might be a book, a parish register, a birth certificate, a census schedule, or...

Whatever it is, the source needs a date and GEDCOM doesn't provide a tag for it, hence John's _DATE tag. So the date for a book is normally the publication date, say 1959. A parish register might have a range 1813-1921  which, if I understand rightly, has to be entered in the _DATE tag as FROM 1813 TO 1921.

Then there's the citation or source detail.

For a book it might just be page 121. You don't really need a date at this point but GEDCOM provides the DATE tag here and, to use Simon's sort method, something had to be entered here for sort to work. It may not now be needed using John's sort.

For a baptism, it's different. The register has a date (or date range), presumably entered in the _DATE tag and the precise baptism has something like

 page 12 entry 103 and the date say, 16 JUL 1861.

So, data entry is, with John's _DATE tag, relatively straightforward. However, getting the info out into a report is not easy at all using standard and even non-standard GEDCOM. The Generations Book report cannot, so far as I can see, produce data held in the source details such as page 161 entry 1234 21 April 1891.

GEDCOM seems utterly incompatible for a system apparently much loved in the US but little used in the UK - the Elizabeth Shown Mills 'Evidence Explained' methodology. I think it needs all sorts of non-standard cludges to make it work.

Similarly, the UK methodology, developed at the University of Strathclyde, is, to my mind, much superior to EE, but it  is still beyond GEDCOM's ability to handle.

In short, I have no idea how to get a properly-crafted citation out of any genealogy software. GCII is as good as any, and better than most, but constrained by the limitations of GEDCOM.

I suppose it would be possible to create a report that could do it using EE, Strathclyde, or my old-fashioned method but well above my pay grade.

In the meantime, if I ever get round to writing all this stuff up properly, I shall have to craft each citation individually.

Cheers

Mick

[Simon Robbins]

That's pretty much what I was saying, although I have seen a possible use for the two date fields in the source record. If for example the source was a say a transcription of a parish register. I suppose date of the source would be when it was transcribed and the date subordinate to the DATA tag in the window opened by the "Content and text from the Source" button would be the date range of the original register. Why else would you use two dates in the record itself? I can see a use for a date in the record and in certain instances one in the citation where the source, such as a baptism register, was not created in one event by one person but was added to many times, by many different people over many years. As for the date in the source record, I'm not sure why standard GEDCOM puts it subordinate to the DATA tag as opposed to John's _DATE tag which is at level 1, but I'm equally not sure how to use both. As for my sorting script, I probably did only look at the date in the citation as that's what I used, it could probably be rewritten but there's no point now as it's a built in function which is much better.

I've never used the Generations Book Report but I would be surprised if it couldn't be tweaked to pick whatever date you wanted as John's scripting options are pretty capable of just about anything, although I'll admit there is a bit of a learning curve, and everything I have done has been completely trial and error, and only when I have had a lot time on my hands! I would suggest if anyone was going to attempt it, it should follow the same methodology as John's sorting and look in all three places.

As for the citation capabilities of any genealogy software, GEDitCOM is probably the only one where you can create new custom tags to do whatever you want, but of course at the expense of not sticking to the GEDCOM standard. Given the most other software doesn't seem to rigorously enforce the standard, I'm not sure that is too important.

Simon

[John Nairn]

Hi Simon,

I thought about making Tree->Sort->Sources menu command, but context was a challenge. It was difficult to determine (across browser styles) the proper context for when sorting makes sense. I therefore used your method of using an Internal Script embedded in the Default Format (it could be added to other broswer styles too, if it makes sense).

Regarding finding the command, I used your method of changing one source icon from attaching a source to be the new sort option (but only when next to sources and not in the “Attach” section, where attaching a new source makes more sense). I agree it is kind of hidden. In fact, I did not even notice you had that option in the Modified Default Format until it was pointed out to me on the group. Another option would be to put a “Sort Sources” button there too.

Regarding GEDCOM source command, keeping the highlighting would not work, but maybe the text color could be changed or some other feature (I do that for corrupted text already). I will look into that.

The reason for different methods for raw GEDCOM display are in software design. The old GEDitCOM just read GEDCOM text and passed blocks of that text to windows. The user interface decoded the text every time to create the display. Because the window had some GEDCOM text, it was easy to toggle back and forth between that text and the display already written in the first draft. The problem with old GEDitCOM was that is was inefficient to rescan through text for each new display and each new feature needed. When I switched to GEDitCOM II, which is a complete rewrite using the rewritten MacOS X methods and object-oriented programming, I changed from storing raw text to decoding into an Objective C classes for GEDCOM structures. Each structure object has level number, GEDCOM tag, and contents and might contain more GEDCOM structures for subordinate data. This approach made it much easier work with GEDCOM data.

When I started on old GEDitCOM, the first draft just showed GEDCOM text so every window was based around some block of text. When I added features to do a custom interface, it was easy to toggle back and forth between GEDCOM text and custom display. When I wrote GEDitCOM II, the first draft just read a file, decoded everything into GEDCOM structure objects, and opened an index window. That window would show GEDCOM text when you clicked on any record and you could edit the text. When I started browser styles for custom displays, they were built on blocks of GEDCOM structures instead of raw text. It would probably be possible (but challenging) to add GEDCOM text editing to those window, but because I already had GEDCOM editing in the index window, I decided to stay with that.

John

To view this discussion on the web visit https://groups.google.com/d/msgid/geditcom-ii-discussions/368b873f-5b2f-44b4-98a1-848d1628cc0dn%40googlegroups.com.

[theboo...@gmail.com]

Hi Simon

Using a non-standard tag may well be a way through here. Not yet sure but, as you say, it's probably okay given that other software ain't so rigorous about the standard anyway. So thanks for the thought.

I've not used the DATA.EVEN.DATE setup and will explore that as well with more of an open mind than when I looked at it before and couldn't see a use for it.

Back in July 2021, John explained why it's hard to get full citation data into the Generations Book and, I assume, into other reports. This is what he said:

The source list in Create Generations Book Report is created from the source records in your file and those records do not have page numbers. The page number you mentioned are in the record that cites the source. The problem is that one source (e.g., a census) might be cited by many individuals and each one would have a different page number. The question is then — what page number should be in the source list?

It might be possible to modify the book output to use page number if one only person cites that sources, but that is not part report now (and might be challenging).

The source list is done using BibTeX option of LaTeX. The most recent change I made to the script (September 2020) was to try and put as much source information as possible into the list. The problem is the GEDCOM sources do not correspond very well to BibTeX source entries. What is done currently is that a different type of BibTeX entry is made depending on what you choose for the type of source in the source’s pop-up menu. For each type, the script tries to find the best match to available fields in BibTeX files. If you enter a page number in any of the available fields, it will appear in the source list. For example, using “Article” type, a page number might make sense ins the “Journal, Periodical, Volume, etc.” field.

I think sources are very important, but they are challenging. While GEDCOM gives lots of options for citing source, it give a lot fewer options for entering information about each source in its source record. I had to create a custom tag for source type and surprisingly even had to create one to reliable document the source's “Date.".

This thread has been most informative. Thanks to all involved.

Cheers

Mick

[Simon Robbins]

Thanks John, for the explanation of how the data is treated differently in GEDitCOM II from the original GEDitCOM which I can sort of understand whilst not being a programmer. It's a shame really as I think from a users perspective the original application behaviour was by far the most elegant solution, even if not the best from a programming perspective.

Simon

[theboo...@gmail.com]

I'm so used to the sort sources option that the icon never bothered me. It is a wee bit hidden though, I suppose. Perhaps it could be changed to something like the attached.

Mick