Family line merges - proposed process
13 views
Skip to first unread message
Bob Fields
Apr 3, 2012, 1:24:58 PM4/3/12
to WikiTree Mayflower Ancestors User Group
I've spent a while merging many people in several long family lines,
and came up with a step by step process described here, before I add
it to http://www.wikitree.com/wiki/Merging_multiple_profiles. Sorry
for the long message, I'm very process-oriented. Comments and feedback
welcome.
I've found between 5 and 20 duplicates for many of my ancestors before
1700. Most of this process doesn't require supervisor access, though
you have to wait for the other profile owners to approve access and
perform the merges. Starting with the oldest ancestor works around the
problem of merging different parents when merging children.
For a family line with many duplicates:
1. Start with the oldest ancestor shown as a duplicate in
http://www.wikitree.com/wiki/Special:FindMatches (your own ancestor
potential matches). I usually search across wikitree, +- 2 years,
require dates, don't allow for century typos, and don't show proposed
or rejected matches. I wish I could set those as defaults to use every
time.
2. Start with the oldest duplicate ancestor in the list first. Show
the ancestor lists for all duplicates, to find the oldest duplicate
person. That person may not be on your watchlist so they won't show up
as a match in the original search. Start with that person / family
group, husband first. First / last names may have different spellings
for the same people.
3. Run WikiTree search with the same name(s) to find ALL duplicates,
because they don't always show up in #2. Since the search is NOT a
soundex or fuzzy match, you may have to search several times with all
name variations. Since firstname/lastname results are NOT sorted by
date and there is no date filtering available, you have to work your
way through all possible duplicates in the entire result list, which
may be very long for common names like Henry Adams or John Wright.
4. Line up all duplicates for a person next to each other (I use tab
windows in the browser) in order of creation (oldest first). Typically
it's by number order after last name, unless there are variations on
the lastname spelling.
5. You may find people with the same name but missing dates. Open
those people too, to see if they are the same person (based on family
relationships). If they are, fill in at least the birth date. It's
amazing the number of people that add profiles without even an
estimated before/after date.
6. Once you have all possible candidates, verify that they really are
the same person, based on birth/marriage/death dates and locations,
parent/spouse/child names and relationships. Close the ones that are
not the same person, making notes of any data issues such as improper
relationships (parents too young, multiple children in the same year,
duplicate children).
7. If you have permission or the profile is 300+ years old or open,
fix any incorrect or conflicting data (dates, locations) in the
duplicate profiles. Don't worry about missing information. If you see
incorrect relationships, don't merge that profile until the
relationships are fixed. If there are multiple variations on the first
or last names, add those variations to the alternate names. Add
missing last names or missing maiden names, if allowed.
8. For each duplicate, request to join the trusted list, This notifies
the profile owners that you are a related descendant, and makes your
access permanent instead of temporary. I paste a stock request such as
'Merging duplicate profiles and adding ancestors/spouse/children. I am
a descendant.' and may add specific information about how I am
descended (through XX child or grandchild) if a historically
significant ancestor or family line. Take control of any orphaned
profiles.
9. From all the candidates, decide which profile will be the surviving
merged profile. For each duplicate profile pair, create a merge
request at http://www.wikitree.com/wiki/Special:MergePerson with the
IDs merging every later profile into the single earliest profile. This
also notifies the profile owners of the specific merge request, and
adds it to the profile history as a pending merge.
10. If you are a supervisor and want to manage the merged profile, you
can add yourself as a profile manager to the single earliest profile,
at the same time merging your own personal ancestry information into
that profile. You can also remove the other profile managers, though I
have not done that on any duplicate profiles. Or you can simply add
yourself as profile manager to both profiles and do the merge
yourself. I do think it is important to at least ask permission and
notify the other profile owners though.
11. I find many owners very responsive and helpful, and they do the
proposed merges sometimes very quickly. Others may take a long time to
respond.
12. When all duplicates for a single person are merged, we can clean
up the description and source info as suggested by Lianne. The merge
process simply adds all text notes, so people can't complain about
losing any information, until we start to clean up the description. I
may clean up some of the text in step #7 if the information is truly
not useful ('imported from gedcom, ancestry', etc).
13. As an additional validation, I periodically export the gedcom and
import it into a program such as RootsWeb or FamilyTreeSearch, and run
the data validations. This is extremely valuable for finding improper
relationships and improbable/impossible dates.
Though the description at http://www.wikitree.com/wiki/Merging_multiple_profiles
is good, it doesn't detail the entire process of merging entire family
trees. Though this is a tedious process, in the end the data and
relationships are much more complete.
and came up with a step by step process described here, before I add
it to http://www.wikitree.com/wiki/Merging_multiple_profiles. Sorry
for the long message, I'm very process-oriented. Comments and feedback
welcome.
I've found between 5 and 20 duplicates for many of my ancestors before
1700. Most of this process doesn't require supervisor access, though
you have to wait for the other profile owners to approve access and
perform the merges. Starting with the oldest ancestor works around the
problem of merging different parents when merging children.
For a family line with many duplicates:
1. Start with the oldest ancestor shown as a duplicate in
http://www.wikitree.com/wiki/Special:FindMatches (your own ancestor
potential matches). I usually search across wikitree, +- 2 years,
require dates, don't allow for century typos, and don't show proposed
or rejected matches. I wish I could set those as defaults to use every
time.
2. Start with the oldest duplicate ancestor in the list first. Show
the ancestor lists for all duplicates, to find the oldest duplicate
person. That person may not be on your watchlist so they won't show up
as a match in the original search. Start with that person / family
group, husband first. First / last names may have different spellings
for the same people.
3. Run WikiTree search with the same name(s) to find ALL duplicates,
because they don't always show up in #2. Since the search is NOT a
soundex or fuzzy match, you may have to search several times with all
name variations. Since firstname/lastname results are NOT sorted by
date and there is no date filtering available, you have to work your
way through all possible duplicates in the entire result list, which
may be very long for common names like Henry Adams or John Wright.
4. Line up all duplicates for a person next to each other (I use tab
windows in the browser) in order of creation (oldest first). Typically
it's by number order after last name, unless there are variations on
the lastname spelling.
5. You may find people with the same name but missing dates. Open
those people too, to see if they are the same person (based on family
relationships). If they are, fill in at least the birth date. It's
amazing the number of people that add profiles without even an
estimated before/after date.
6. Once you have all possible candidates, verify that they really are
the same person, based on birth/marriage/death dates and locations,
parent/spouse/child names and relationships. Close the ones that are
not the same person, making notes of any data issues such as improper
relationships (parents too young, multiple children in the same year,
duplicate children).
7. If you have permission or the profile is 300+ years old or open,
fix any incorrect or conflicting data (dates, locations) in the
duplicate profiles. Don't worry about missing information. If you see
incorrect relationships, don't merge that profile until the
relationships are fixed. If there are multiple variations on the first
or last names, add those variations to the alternate names. Add
missing last names or missing maiden names, if allowed.
8. For each duplicate, request to join the trusted list, This notifies
the profile owners that you are a related descendant, and makes your
access permanent instead of temporary. I paste a stock request such as
'Merging duplicate profiles and adding ancestors/spouse/children. I am
a descendant.' and may add specific information about how I am
descended (through XX child or grandchild) if a historically
significant ancestor or family line. Take control of any orphaned
profiles.
9. From all the candidates, decide which profile will be the surviving
merged profile. For each duplicate profile pair, create a merge
request at http://www.wikitree.com/wiki/Special:MergePerson with the
IDs merging every later profile into the single earliest profile. This
also notifies the profile owners of the specific merge request, and
adds it to the profile history as a pending merge.
10. If you are a supervisor and want to manage the merged profile, you
can add yourself as a profile manager to the single earliest profile,
at the same time merging your own personal ancestry information into
that profile. You can also remove the other profile managers, though I
have not done that on any duplicate profiles. Or you can simply add
yourself as profile manager to both profiles and do the merge
yourself. I do think it is important to at least ask permission and
notify the other profile owners though.
11. I find many owners very responsive and helpful, and they do the
proposed merges sometimes very quickly. Others may take a long time to
respond.
12. When all duplicates for a single person are merged, we can clean
up the description and source info as suggested by Lianne. The merge
process simply adds all text notes, so people can't complain about
losing any information, until we start to clean up the description. I
may clean up some of the text in step #7 if the information is truly
not useful ('imported from gedcom, ancestry', etc).
13. As an additional validation, I periodically export the gedcom and
import it into a program such as RootsWeb or FamilyTreeSearch, and run
the data validations. This is extremely valuable for finding improper
relationships and improbable/impossible dates.
Though the description at http://www.wikitree.com/wiki/Merging_multiple_profiles
is good, it doesn't detail the entire process of merging entire family
trees. Though this is a tedious process, in the end the data and
relationships are much more complete.
Lianne Lavoie
Apr 3, 2012, 3:43:50 PM4/3/12
to wikima...@googlegroups.com
Thanks for doing this, Bob! :) I'm gonna pick at it, but just because I'm really picky. Overall this is great!
One thing I'd question on here: why take the time to correct data in the individual profiles before merging them? As long as one profile has it right, you can end up with the right info just by selecting the right ones during merging. And even if not, it's easier to change the data at the end when you have just one profile. I have the same issue, though to a lesser extent, with not merging profiles until relationships are fixed. I guess it depends on which relationships. Parents are easily fixed while merging, just by selecting the right ones. Children, I would tend to worry about after finishing the parents, when I'm ready to start on the next generation. I guess spouses are equally easy to remove before or after a merge.
I wasn't sure what you meant by this part: "For each duplicate, request to join the trusted list, This notifies the profile owners that you are a related descendant, and makes your access permanent instead of temporary." What do you mean by permanent vs. temporary?
The other part that confused me a bit was this: "If you are a supervisor and want to manage the merged profile, you can add yourself as a profile manager to the single earliest profile, at the same time merging your own personal ancestry information into that profile." It's that last bit about merging your own personal ancestry information into the profile that I don't understand. Also, keep in mind that anyone who's a manager of any of the merged profiles will be a manager of the final one, so it doesn't really matter which one you add yourself to.
~Lianne--
One thing I'd question on here: why take the time to correct data in the individual profiles before merging them? As long as one profile has it right, you can end up with the right info just by selecting the right ones during merging. And even if not, it's easier to change the data at the end when you have just one profile. I have the same issue, though to a lesser extent, with not merging profiles until relationships are fixed. I guess it depends on which relationships. Parents are easily fixed while merging, just by selecting the right ones. Children, I would tend to worry about after finishing the parents, when I'm ready to start on the next generation. I guess spouses are equally easy to remove before or after a merge.
I wasn't sure what you meant by this part: "For each duplicate, request to join the trusted list, This notifies the profile owners that you are a related descendant, and makes your access permanent instead of temporary." What do you mean by permanent vs. temporary?
The other part that confused me a bit was this: "If you are a supervisor and want to manage the merged profile, you can add yourself as a profile manager to the single earliest profile, at the same time merging your own personal ancestry information into that profile." It's that last bit about merging your own personal ancestry information into the profile that I don't understand. Also, keep in mind that anyone who's a manager of any of the merged profiles will be a manager of the final one, so it doesn't really matter which one you add yourself to.
~Lianne
Lianne Lavoie, BCSc
An Effort in Green
An Effort in Green
Chris Whitten
Apr 3, 2012, 5:10:53 PM4/3/12
to wikima...@googlegroups.com
I'm with Lianne. Thanks for doing this, Bob.
A side note: We're working an improved person search engine. First
we'll just be using it to replace
http://dev3.wikitree.com/wiki/Special:SearchPerson but soon after that
we can look at using it to improve FindMatches.
Chris
Chris Whitten
Apr 3, 2012, 5:14:04 PM4/3/12
to wikima...@googlegroups.com
Oops, I put the wrong URL in my message. (I put the one for our
development server, where we work on changes.) The URL for the current
person search engine is
http://www.wikitree.com/wiki/Special:SearchPerson
development server, where we work on changes.) The URL for the current
person search engine is
http://www.wikitree.com/wiki/Special:SearchPerson
The one we're working on will be much better at finding people with
variations in all the name fields.
Chris
On Tue, Apr 3, 2012 at 5:10 PM, Chris Whitten <cwhi...@gmail.com> wrote:
> I'm with Lianne. Thanks for doing this, Bob.
>
> A side note: We're working an improved person search engine. First
> we'll just be using it to replace
> http://dev3.wikitree.com/wiki/Special:SearchPerson but soon after that
> we can look at using it to improve FindMatches.
>
> Chris
--
Chris Whitten, WikiTreer-in-Chief
http://www.WikiTree.com/wiki/Contact_us#Chris
Bob Fields
Apr 4, 2012, 9:58:04 AM4/4/12
to wikima...@googlegroups.com, ch...@wikitree.com
Here's the search engine I use the most:
http://wc.rootsweb.ancestry.com/cgi-bin/igm.cgi
I like the 'has notes' and 'has sources' options, I would also add 'has parents' because I am usually trying to extend the line further backwards or add additional supporting documentation.
I use it to resolve conflicting dates and locations during merges. 300+ years back I always see duplicate entries, I look for the ones with the best documentation, most exact dates, most siblings, largest pedigree.
If you could add 'sort by birth date' in the results when searching on first/last name, that would also be helpful.
On Tuesday, April 3, 2012 5:14:04 PM UTC-4, Chris Whitten wrote:
http://wc.rootsweb.ancestry.com/cgi-bin/igm.cgi
I like the 'has notes' and 'has sources' options, I would also add 'has parents' because I am usually trying to extend the line further backwards or add additional supporting documentation.
I use it to resolve conflicting dates and locations during merges. 300+ years back I always see duplicate entries, I look for the ones with the best documentation, most exact dates, most siblings, largest pedigree.
If you could add 'sort by birth date' in the results when searching on first/last name, that would also be helpful.
On Tuesday, April 3, 2012 5:14:04 PM UTC-4, Chris Whitten wrote:
Oops, I put the wrong URL in my message. (I put the one for our
development server, where we work on changes.) The URL for the current
person search engine is
http://www.wikitree.com/wiki/Special:SearchPersonThe one we're working on will be much better at finding people with
variations in all the name fields.Chris
Bob Fields
Apr 4, 2012, 10:13:25 AM4/4/12
to wikima...@googlegroups.com
I thought you had only temporary access to change a profile (5 days) when you requested a merge. Perhaps that access is as a temporary profile manager, to add/update child/spouse/parent relationships, so the trusted access is unnecessary. Anyway, it only takes a second to request access.
I correct the conflicting data that is wrong in the individual profiles because I don't always do the merges myself. The request is sent to the profile owner(s) and they approve and do the merge. That way, there is no question about which date or location survives in the merged profile, in case they override the default that is retained in the older profile.
I merge my own information in because at this point I am only working with my own direct ancestors and their near relations. There's plenty of work to be done in those, for me. I typically want to manage those profiles myself, though I may remove myself afterwards for distantly related profiles. If you are helping someone else then you don't need to do that. For fixing relationships, I was talking about conflicting or incorrect relationships rather than missing relationships - I see a few instances of the same person with different parents, etc.
It would be nice to have some additional level of access (lower than trusted), where anybody can add missing spouse/children/parents/dates/locations, but not change or merge the data that is already there entered by somebody else. Right now you need to be a profile owner to do that. That's the opposite of the current state for 300+ year old profiles. The profile owner would be notified, and can always remove the info if incorrect, but I think most people would be happy to have others fill in missing info.
On Tuesday, April 3, 2012 3:43:50 PM UTC-4, Lianne Lavoie wrote:
I correct the conflicting data that is wrong in the individual profiles because I don't always do the merges myself. The request is sent to the profile owner(s) and they approve and do the merge. That way, there is no question about which date or location survives in the merged profile, in case they override the default that is retained in the older profile.
I merge my own information in because at this point I am only working with my own direct ancestors and their near relations. There's plenty of work to be done in those, for me. I typically want to manage those profiles myself, though I may remove myself afterwards for distantly related profiles. If you are helping someone else then you don't need to do that. For fixing relationships, I was talking about conflicting or incorrect relationships rather than missing relationships - I see a few instances of the same person with different parents, etc.
It would be nice to have some additional level of access (lower than trusted), where anybody can add missing spouse/children/parents/dates/locations, but not change or merge the data that is already there entered by somebody else. Right now you need to be a profile owner to do that. That's the opposite of the current state for 300+ year old profiles. The profile owner would be notified, and can always remove the info if incorrect, but I think most people would be happy to have others fill in missing info.
On Tuesday, April 3, 2012 3:43:50 PM UTC-4, Lianne Lavoie wrote:
Thanks for doing this, Bob! :) I'm gonna pick at it, but just because I'm really picky. Overall this is great!
One thing I'd question on here: why take the time to correct data in the individual profiles before merging them? As long as one profile has it right, you can end up with the right info just by selecting the right ones during merging. And even if not, it's easier to change the data at the end when you have just one profile. I have the same issue, though to a lesser extent, with not merging profiles until relationships are fixed. I guess it depends on which relationships. Parents are easily fixed while merging, just by selecting the right ones. Children, I would tend to worry about after finishing the parents, when I'm ready to start on the next generation. I guess spouses are equally easy to remove before or after a merge.
I wasn't sure what you meant by this part: "For each duplicate, request to join the trusted list, This notifies the profile owners that you are a related descendant, and makes your access permanent instead of temporary." What do you mean by permanent vs. temporary?
The other part that confused me a bit was this: "If you are a supervisor and want to manage the merged profile, you can add yourself as a profile manager to the single earliest profile, at the same time merging your own personal ancestry information into that profile." It's that last bit about merging your own personal ancestry information into the profile that I don't understand. Also, keep in mind that anyone who's a manager of any of the merged profiles will be a manager of the final one, so it doesn't really matter which one you add yourself to.
~Lianne
Becky
Apr 4, 2012, 11:32:37 AM4/4/12
to wikima...@googlegroups.com
Bob, Congratulations on becoming manager. I'm glad you did this write up, thank you.
The advice to notify the managers ahead of the merge is very important, as is encouraging a
lot of analysis of all the relationships in the profile before merging. (Merging sometimes
gives me a panic attack for fear of loosing someone's work, etc.)
Reply all
Reply to author
Forward
0 new messages