FW: End of Term web harvest of US govt sites

9 views
Skip to first unread message

Patrice McDermott

unread,
Nov 4, 2016, 11:08:11 AM11/4/16
to openthego...@googlegroups.com, no-free-lunch-caucus, secrecy...@lists.sonic.net, ogp--us-civil-soci...@googlegroups.com, us-open-g...@googlegroups.com, surveillanc...@googlegroups.com, crimmi...@googlegroups.com, lawenforcemen...@googlegroups.com, jrja...@stanford.edu

Please find below information on an end-of-term US govt web crawl.  I, for one, am really happy to know this is occurring. LOTS of stuff went missing (was disappeared) in the switch from Clinton(1) to GW Bush....

 

The drivers for the project are the Library of Congress, GPO, and the library community. All folks who know how to do this sort of thing....

 

There's more information about the collaborative project in this blog post, Nominations Sought for the U.S. Federal Government End of Term Web Archive 

 

The project has a good ‘seed list’ of .gov & .mil sites and  is looking, especially, for sites that are on .com.  The places to nominate them are linked below.  It is up to you to find them.....

 

Please excuse the inevitable duplicate postings.

 

Patrice

 

Patrice McDermott, Executive Director

OpenTheGovernment.org

202.332.6736

 

---------- Forwarded message ----------
From: Mr. James R. (Librarian) Jacobs <jrja...@stanford.edu>
Date: Thu, Oct 27, 2016 at 11:53 AM
Subject: End of Term archive seed nominations
To: DIG-PRES...@unt.edu

Hi all,

We’re making good progress on the 2016 End of Term crawl of the .gov domain and will soon begin to crawl and preserve the domain. We have a solid seed list for the .gov domain. However, there are a lot of govt entities which have their sites on .com instead of .gov or .mil domains. We’d like to ask for your help in finding and nominating those sites for the crawl. If you know of any, please use the nomination tool to add them to our seed list. Please let me know if you have further questions.

http://digital2.library.unt.edu/nomination/eth2016/

http://digital2.library.unt.edu/nomination/eth2016/add/

We greatly appreciate your help.

best,

James Jacobs


--
James R. Jacobs
US Government Information Librarian
123D Green Library
Stanford University
P: 650.862.9871
E: jrja...@stanford.edu
AIM: LibrarianJames
T: @freegovinfo
W: freegovinfo.info
lockss-usdocs.stanford.edu
library.stanford.edu/people/jrjacobs

"The art of research is the ability to look at the details, and see the passion."
-- Daryl Zero, "The Zero Effect" (1998)
----------------------------------
This message has been intercepted and read by U.S. government agencies including the FBI, CIA, and NSA without notice, warrant or knowledge of sender or recipient and without regard for the US Constitution.

(\
{|||8-
(/

 

Zarek, Corinna J. EOP/OSTP

unread,
Nov 4, 2016, 12:54:03 PM11/4/16
to Patrice McDermott, us-open-g...@googlegroups.com

This group may also be interested in the recent blog post about preserving the President’s digital history with the National Archives.

 

https://www.whitehouse.gov/blog/2016/10/31/digital-transition-how-presidential-transition-works-social-media-age

 

The Digital Transition: How the Presidential Transition Works in the Social Media Age

OCTOBER 31, 2016 AT 4:00 PM ET BY KORI SCHULMAN

TWITTER FACEBOOK EMAIL 

Summary: 

Take a look at how we plan to preserve and pass on the digital history of the Obama administration.

President Obama on his iPhone

President Obama is the first “social media president”: the first to have @POTUS on Twitter, the first to go live on Facebook from the Oval Office, the first to answer questions from citizens on YouTube, the first to use a filter on Snapchat. Over the past eight years, the President, Vice President, First Lady, and the White House have used social media and technology to engage with people around the country and the world on the most important issues of our time (while having some fun along the way). 

Looking back over the past eight years, our digital footprint reflects some broader changes in the ways people consume news and information and engage with the world around them online. In 2009, in addition to rolling out a revamped WhiteHouse.gov featuring a blog, RSS, and an email list, we joined Facebook, Twitter, Flickr, Vimeo, iTunes, and even MySpace. In 2011, we launched We the People, a platform for citizens to petition the White House. In 2013, the First Lady posted her first photo to Instagram. In 2015, President Obama sent his first tweet from @POTUS. In 2016, the White House debuted an official story on Snapchat for the State of the Union. This digital infrastructure is an asset not just for the next president but for all future presidents to build off of. The archive belongs to the American people.

From the very beginning, our mission has been to reach Americans and people around the world on the channels and platforms where they already spend their time. This work began on President Obama’s 2008 campaign and, over the course of this Administration, has increasingly meant meeting people where they are online, using technology to re-imagine traditional formats, and creating unique opportunities for people to interact with their government.

The past eight years have left us with tremendous optimism about how future administrations might use these tools to create even more pathways for meaningful civic participation. 

So after all this – nearly eight years of digital firsts, more than 470,000 We the Peoplepetitions, nearly 30,000 @WhiteHouse tweets, and thousands of hours of video footage – what happens when the next administration takes office? 

The President has made clear that a smooth transition between administrations is one of his top priorities, and digital is a key component of that effort. While much of the digital transition is unprecedented in the United States, the peaceful transition of power is not. There are some important principles we can work from. Accordingly, we have been working to ensure that our digital transition meets three key goals. First, we are preserving the material we’ve created with the National Archives and Records Administration (NARA). From tweets to snaps, all of the material we’ve published online will be preserved with NARA just as previous administrations have done with records ranging from handwritten notes to faxes to emails. Second, wherever possible, we are working to ensure these materials continue to be accessible on the platforms where they were created, allowing for real time access to the content we’ve developed. Finally, we are working to ensure that the next president and administration – regardless of party – can continue to use and develop the digital assets we have created to connect directly with the people they serve. 

Here is an overview of our plans for how this will work:

Social Media:

On Twitter, for example, the handle @POTUS will be made available to the 45th President of the United States on January 20, 2017. The account will retain its more than 11 million followers, but start with no tweets on the timeline. @POTUS44, a newly created handle maintained by NARA, will contain all of President Obama’s tweets and will be accessible to the public on Twitter as an archive of President Obama’s use of the account. In addition, President Obama’s tweets will also be archived at NARA, where they will be preserved and accessible in the same manner as all other Presidential records. This will also be the case for other Twitter handles, including @WhiteHouse, @FLOTUS, @PressSec, and @VP. Individual official accounts, like @KS44, will transition to NARA and continue to be accessible for the public to view as an archive of all tweets sent by White House staff. 

On Instagram and Facebook, the incoming White House will gain access to the White House username, URL, and retain the followers, but will start with no content on the timeline. An archive of White House content that was posted to the Obama White House Instagram and Facebook will continue to be accessible to the public at Instagram.com/ObamaWhiteHouse and Facebook.com/ObamaWhiteHouse. Facebook accounts for President Obama and the Vice President and the Instagram accounts belonging to the First Lady and Vice President will be moved to new “44” usernames and preserved by NARA. 

We’ll follow a similar approach with other official accounts on platforms including Medium, Tumblr, and YouTube. These presences will be made available to the 45th White House, including the “White House” username, /WhiteHouse URL, and the followers, but start with no content on the accounts. The Obama White House content will be preserved and accessible in the same manner as all other presidential records and continue to be available on the platform at a new URL. 

We the People:

Today, the White House’s We The People website has more than 12 million verified users that have created more than 470,000 petitions to their government on the issues they care the most about. The White House mobilizes government officials to respond to every petition that receives at least 100,000 signatures. Over the years, we’ve seen it become a powerful platform, with petitions reaching senior staff and the President’s desk and even shaping policy on topics including cell phone unlocking and net neutrality. The We the People code has been open-sourced and we’re taking every step possible to make it easy for future administrations to carry on this tradition. The petitions and our responses will also be archived with NARA.  

WhiteHouse.gov:

Similar to the Clinton and Bush White House websites, President Obama’s WhiteHouse.gov will be preserved on the web and frozen after January 20th and made available at ObamaWhiteHouse.gov. The incoming White House will receive the WhiteHouse.gov domain and all content that has been posted to WhiteHouse.gov during the Obama administration will be archived with NARA. 

Video and Photos:

Over the past eight years, we’ve racked up thousands of hours of video footage and millions of photos. That’s all being transferred to NARA, where it will be preserved and made available to the public pursuant to the Presidential Records Act. The photos that have been released on WhiteHouse.gov, Flickr, Instagram and the video content on YouTube, Vimeo, and other White House presences will continue to be available on these platforms with updated usernames and will also be maintained by NARA. Full resolution versions of all the photo and video content that has been posted online will also be archived with NARA. 

Opening Up the Data to the Public:

In addition to the steps that the White House and NARA are taking, we want to open up this process to the American people. Specifically, by the end of this Administration, we're committed to publicly sharing our social media content in an easily accessible and comprehensive way (e.g., zip files to download). In the interim, we're inviting the American public – from students and data engineers, to artists and researchers – to come up with creative ways to archive this content and make it both useful and available for years to come. From Twitter bots and art projects to printed books and query tools, we’re open to it all. The White House will make our social media data available early to people who are interested in building something for the public. For more information on how to submit an idea, click here

SUBMIT YOUR IDEA

Given the unprecedented nature and scale of the digital transition, we anticipate we’ll learn a lot along the way – so these plans may evolve. You can follow along @WHWeb for updates on the digital transition process, data releases, and more. 

--
You received this message because you are subscribed to the Google Groups "US Open Government" group.
To unsubscribe from this group and stop receiving emails from it, send an email to us-open-governm...@googlegroups.com.
To post to this group, send email to us-open-g...@googlegroups.com.
Visit this group at https://groups.google.com/group/us-open-government.
For more options, visit https://groups.google.com/d/optout.

Reply all
Reply to author
Forward
0 new messages