Sync failing

[Ryan Bis]

Maybe I'm not doing something right, but when I run repo sync, it keeps failing.

error: Exited sync due to fetch errors.

I see a lot of 403 errors while it's running too.

Should I just keep retrying, or is there something I may have to fix on my end? (Note: I've always seemed to have problems syncing, but I can't get it to work this time. I currently have r16, and I'm trying to update to r17).

Ryan

[Tor Norbye]

For what it's worth, checking out a full repository from scratch
worked for me last week. I'm about to check out another one (once I've
freed up some more diskspace!) so I'll let you know if I see any new
again.

-- Tor

[Raphaël Moll]

On Sat, Mar 31, 2012 at 12:46 PM, Ryan Bis <impi...@gmail.com> wrote:

Works here and I sync every day.

What's the exact command-line you're using?

R/

[Ryan Bis]

Sorry, I should have posted here. After running repo sync -f a few more times, it finally finished. Although I'm not sure it actually got anything, I still saw a lot of 403 errors while it was running. I'll try again today to see if I get better results.

I'm still getting errors in Eclipse when trying to import the projects, which leads me to believe that maybe the whole source didnt sync properly. (I ran through the steps on the tools website).

[Raphaël Moll]

On Mon, Apr 2, 2012 at 3:50 PM, Ryan Bis <impi...@gmail.com> wrote:

Sorry, I should have posted here. After running repo sync -f a few more times, it finally finished. Although I'm not sure it actually got anything, I still saw a lot of 403 errors while it was running. I'll try again today to see if I get better results.

I'm still getting errors in Eclipse when trying to import the projects, which leads me to believe that maybe the whole source didnt sync properly. (I ran through the steps on the tools website).

Suggestion: You might want to give us some specifics if you want some actual help.

R/

[Ryan Bis]

Your right, here's the output from repo sync:

repo sync
Fetching projects:  76% (188/247)  error: Couldn't resolve host 'android.googlesource.com' while accessing https://android.googlesource.com/platform/packages/apps/Gallery/info/refs

fatal: HTTP request failed
error: Couldn't resolve host 'android.googlesource.com' while accessing https://android.googlesource.com/platform/packages/apps/Exchange/info/refs

fatal: HTTP request failed
error: Couldn't resolve host 'android.googlesource.com' while accessing https://android.googlesource.com/platform/packages/apps/Gallery/info/refs

fatal: HTTP request failed
error: The requested URL returned error: 403 while accessing https://android.googlesource.com/platform/packages/apps/Exchange/info/refs

fatal: HTTP request failed
error: Failed connect to android.googlesource.com:443; Connection timed out while accessing https://android.googlesource.com/platform/packages/apps/Gallery2/info/refs

fatal: HTTP request failed
error: Cannot fetch platform/packages/apps/Gallery
Fetching projects:  77% (191/247)  error: Cannot fetch platform/packages/apps/Exchange
Fetching projects:  78% (193/247) 

error: Exited sync due to fetch errors

If I use repo sync -f, it ends fine, but still throws those HTTP errors. I don't think those are needed for importing the ADT sources, but I could be wrong.

I'm running Ubuntu 11.1 in a VM. It's 32 bit. I had r16 imported fine without errors. Before upgrading, I backed up the r16 projects, then deleted them from the workspace.

Now, it's giving me a bunch of errors about classes not being resolved. This is from the problem log:

Description    Resource    Path    Location    Type
Bundle 'com.android.ide.eclipse.base' cannot be resolved    MANIFEST.MF    /adt/META-INF    line 21    Plug-in Problem
Bundle 'com.android.ide.eclipse.base' cannot be resolved    MANIFEST.MF    /ddms-plugin/META-INF    line 15    Plug-in Problem
InstallDetails cannot be resolved    LogCatPreferencePage.java    /ddms-plugin/src/com/android/ide/eclipse/ddms/preferences    line 64    Java Problem
InstallDetails cannot be resolved    PreferencePage.java    /ddms-plugin/src/com/android/ide/eclipse/ddms/preferences    line 75    Java Problem
NetworkPanel cannot be resolved to a type    NetworkStatisticsView.java    /ddms-plugin/src/com/android/ide/eclipse/ddms/views    line 26    Java Problem
NetworkPanel cannot be resolved to a type    NetworkStatisticsView.java    /ddms-plugin/src/com/android/ide/eclipse/ddms/views    line 33    Java Problem
NetworkPanel cannot be resolved to a type    NetworkStatisticsView.java    /ddms-plugin/src/com/android/ide/eclipse/ddms/views    line 33    Java Problem
NetworkPanel cannot be resolved to a type    NetworkStatisticsView.java    /ddms-plugin/src/com/android/ide/eclipse/ddms/views    line 34    Java Problem
NetworkPanel cannot be resolved to a type    NetworkStatisticsView.java    /ddms-plugin/src/com/android/ide/eclipse/ddms/views    line 36    Java Problem
NetworkPanel cannot be resolved to a type    NetworkStatisticsView.java    /ddms-plugin/src/com/android/ide/eclipse/ddms/views    line 39    Java Problem
NetworkPanel cannot be resolved to a type    NetworkStatisticsView.java    /ddms-plugin/src/com/android/ide/eclipse/ddms/views    line 44    Java Problem
Package 'com.android.tools.lint.client.api' does not exist in this plug-in    MANIFEST.MF    /adt/META-INF    line 121    Plug-in Problem
Project 'adt-tests' is missing required library: '/base/libs/common.jar'    adt-tests        Build path    Build Path Problem
Project 'adt-tests' is missing required library: '/base/libs/sdklib.jar'    adt-tests        Build path    Build Path Problem
The import com.android.ddmuilib.net cannot be resolved    NetworkStatisticsView.java    /ddms-plugin/src/com/android/ide/eclipse/ddms/views    line 19    Java Problem
The import com.android.ide.eclipse.base cannot be resolved    LogCatPreferencePage.java    /ddms-plugin/src/com/android/ide/eclipse/ddms/preferences    line 21    Java Problem
The import com.android.ide.eclipse.base cannot be resolved    PreferencePage.java    /ddms-plugin/src/com/android/ide/eclipse/ddms/preferences    line 21    Java Problem
The project cannot be built until build path errors are resolved    adt-tests        Unknown    Java Problem
The project was not built since it depends on adt-tests, which has build path errors    External Plug-in Libraries        Unknown    Java Problem
The project was not built since its build path is incomplete. Cannot find the class file for com.android.sdklib.IAndroidTarget. Fix the build path then try building this project    adt        Unknown    Java Problem
The type com.android.sdklib.IAndroidTarget cannot be resolved. It is indirectly referenced from required .class files    NewXmlFileCreationPage.java    /adt/src/com/android/ide/eclipse/adt/internal/wizards/newxmlfile    line 1    Java Problem


I've made sure that everything is using Java 6. I noticed in the adt package, there are some jar files in the libs folder which are't added to the build path. Should I add these, or leave it as is?

These jars not in the build path:
androidprefs.jar
commons-codec-1.4.jar
commons-compress-1.0.jar
commons-logging-1.1.1.jar
httpclient-4.1.1.jar
httpmine-4.1.1.jar
sdklib.jar
sdkstats.jar

Let me know if there's anything else I could provide to try to figure this out.

Thanks

On Monday, April 2, 2012 7:40:12 PM UTC-4, Raphaël wrote:

[Raphaël Moll]

On Mon, Apr 2, 2012 at 5:10 PM, Ryan Bis <impi...@gmail.com> wrote:

Your right, here's the output from repo sync:

repo sync
Fetching projects:  76% (188/247)  error: Couldn't resolve host 'android.googlesource.com' while accessing https://android.googlesource.com/platform/packages/apps/Gallery/info/refs

fatal: HTTP request failed
error: Couldn't resolve host 'android.googlesource.com' while accessing https://android.googlesource.com/platform/packages/apps/Exchange/info/refs

fatal: HTTP request failed
error: Couldn't resolve host 'android.googlesource.com' while accessing https://android.googlesource.com/platform/packages/apps/Gallery/info/refs

fatal: HTTP request failed
error: The requested URL returned error: 403 while accessing https://android.googlesource.com/platform/packages/apps/Exchange/info/refs

fatal: HTTP request failed
error: Failed connect to android.googlesource.com:443; Connection timed out while accessing https://android.googlesource.com/platform/packages/apps/Gallery2/info/refs

fatal: HTTP request failed
error: Cannot fetch platform/packages/apps/Gallery
Fetching projects:  77% (191/247)  error: Cannot fetch platform/packages/apps/Exchange
Fetching projects:  78% (193/247) 

error: Exited sync due to fetch errors

If I use repo sync -f, it ends fine, but still throws those HTTP errors. I don't think those are needed for importing the ADT sources, but I could be wrong.

You're doubly right. These aren't needed, but otoh it should work. However I can't help much here. Gerrit has a public mailing list somewhere if you want to ask where, if you want I can dig which one -- I think it's listed on source.android.com.

Siva added a new "base" plugin.

You want to import the Eclipse project for sdk/eclipse/plugins/com.android.ide.eclipse.base -- sorry we forgot to update the documentation too, I'll get to it tomorrow.

That should solve the Eclipse errors. Let me know if it doesn't.

R/

[Raphaël Moll]

Oh I forgot to reply to the "these jars are missing": they will be added to adt as soon as you run the sdk/eclipse/scripts/create_all_symlinks.sh script. It will build the "minimum" for your platform and put the necessary libs in adt. If Eclipse is open while you build these libs, I had issues with it sometimes not realizing a refresh was necessary.

You said earlier you're running java6 in 32 bit. I believe the makefiles now require a 64-bit system -- if you see that, a trick is to simply edit the $(error ..) in the makefile where make reports the error. I did that at home on my shuttle xs35 debian box and it worked like a charm. For building a full blown android that may be an issue, but for adt it doesn't matter whether you're in 32 or 64 bits.

R/

[Ryan Bis]

Raphael,

 Where is the Makefile? The one at the top level only has 3 lines in it. That seems to be the problem, as I'm in 32 bit. I suppose I should just reinstall 64bit Ubuntu rather than dealing with this every time a new version is put out. Thanks for the help.


On Tuesday, April 3, 2012 12:16:45 AM UTC-4, Raphaël wrote:

On Mon, Apr 2, 2012 at 9:08 PM, Raphaël Moll wrote:

[Raphaël Moll]

On Tue, Apr 3, 2012 at 11:02 AM, Ryan Bis <impi...@gmail.com> wrote:

Did you add the base plugin to your Eclipse workspace? Did that solve the errors you showed us yesterday?

 

Raphael,

 Where is the Makefile? The one at the top level only has 3 lines in it. That seems to be the problem, as I'm in 32 bit. I suppose I should just reinstall 64bit Ubuntu rather than dealing with this every time a new version is put out. Thanks for the help.

Did you run  sdk/eclipse/scripts/create_all_symlinks.sh and _actually_ get an error message stating it wouldn't build? If so, then paste the error. If you don't get that error, don't try to fix it. I apologize if my attempt at being exhaustive resulted in more confusion.

R/

[Ryan Bis]

Yes I did. And no, the errors didn't go away. In fact, there are a lot more now.

Here's what I get when running create symlinks:

PLATFORM_VERSION_CODENAME=AOSP
PLATFORM_VERSION=4.0.4.0.4.0.4
TARGET_PRODUCT=sdk
TARGET_BUILD_VARIANT=eng
TARGET_BUILD_TYPE=release
TARGET_BUILD_APPS=
TARGET_ARCH=arm
TARGET_ARCH_VARIANT=armv7-a
HOST_ARCH=x86
HOST_OS=linux
HOST_OS_EXTRA=Linux-3.0.0-12-generic-i686-athlon-with-Ubuntu-11.10-oneiric
HOST_BUILD_TYPE=release
BUILD_ID=OPENMASTER
OUT_DIR=out
============================================
host StaticLib: libhost (out/host/linux-x86/obj/STATIC_LIBRARIES/libhost_intermediates/libhost.a)
ERROR: prebuilts/tools/gcc-sdk/../../gcc/linux-x86/host/x86_64-linux-glibc2.7-4.6/bin/x86_64-linux-ar only run on 64-bit linux
make: *** [out/host/linux-x86/obj/STATIC_LIBRARIES/libhost_intermediates/libhost.a] Error 1


I couldn't find where the makefile for this is. I'm still going to install 64bit so I don't have to deal with things like this in the future, as it seems you're rightly pushing everything towards 64bit. If you want me to try a couple things before I do this, let me know.


On Tuesday, April 3, 2012 2:14:52 PM UTC-4, Raphaël wrote:

[Raphaël Moll]

On Tue, Apr 3, 2012 at 11:26 AM, Ryan Bis <impi...@gmail.com> wrote:

Yes I did. And no, the errors didn't go away. In fact, there are a lot more now.

Here's what I get when running create symlinks:

PLATFORM_VERSION_CODENAME=AOSP
PLATFORM_VERSION=4.0.4.0.4.0.4
TARGET_PRODUCT=sdk
TARGET_BUILD_VARIANT=eng
TARGET_BUILD_TYPE=release
TARGET_BUILD_APPS=
TARGET_ARCH=arm
TARGET_ARCH_VARIANT=armv7-a
HOST_ARCH=x86
HOST_OS=linux
HOST_OS_EXTRA=Linux-3.0.0-12-generic-i686-athlon-with-Ubuntu-11.10-oneiric
HOST_BUILD_TYPE=release
BUILD_ID=OPENMASTER
OUT_DIR=out
============================================
host StaticLib: libhost (out/host/linux-x86/obj/STATIC_LIBRARIES/libhost_intermediates/libhost.a)
ERROR: prebuilts/tools/gcc-sdk/../../gcc/linux-x86/host/x86_64-linux-glibc2.7-4.6/bin/x86_64-linux-ar only run on 64-bit linux
make: *** [out/host/linux-x86/obj/STATIC_LIBRARIES/libhost_intermediates/libhost.a] Error 1

Ah, that's a new one to me.

 

I couldn't find where the makefile for this is. I'm still going to install 64bit so I don't have to deal with things like this in the future, as it seems you're rightly pushing everything towards 64bit. If you want me to try a couple things before I do this, let me know.

Actually it's good you're doing so since I can't work-around that one.  The error means it's using a toolchain that requires a 64-bit system, so my little trick of bypassing the 64-bit check on windows will not help you here.

Sorry for all the trouble.

R/

[Raphaël Moll]

On Mon, Apr 2, 2012 at 5:10 PM, Ryan Bis <impi...@gmail.com> wrote:

Your right, here's the output from repo sync:

repo sync
Fetching projects:  76% (188/247)  error: Couldn't resolve host 'android.googlesource.com' while accessing https://android.googlesource.com/platform/packages/apps/Gallery/info/refs

fatal: HTTP request failed
error: Couldn't resolve host 'android.googlesource.com' while accessing https://android.googlesource.com/platform/packages/apps/Exchange/info/refs

fatal: HTTP request failed
error: Couldn't resolve host 'android.googlesource.com' while accessing https://android.googlesource.com/platform/packages/apps/Gallery/info/refs

fatal: HTTP request failed
error: The requested URL returned error: 403 while accessing https://android.googlesource.com/platform/packages/apps/Exchange/info/refs

fatal: HTTP request failed
error: Failed connect to android.googlesource.com:443; Connection timed out while accessing https://android.googlesource.com/platform/packages/apps/Gallery2/info/refs

fatal: HTTP request failed
error: Cannot fetch platform/packages/apps/Gallery
Fetching projects:  77% (191/247)  error: Cannot fetch platform/packages/apps/Exchange
Fetching projects:  78% (193/247) 

error: Exited sync due to fetch errors

Coming back to the 403 issue, I asked around and I've been told one issue is that repo sync tries to do multiple connections at once and fails on some proxies. You can try to do a "repo sync -j 1" so that it fetches only one project at a time. Feel free to report if you try that and whether it helps or not -- I haven't tried it myself [*].

[*] On Windows with cygwin, I /have/ to use -j 1 otherwise cygwin outputs a bunch of "fork fatal errors" (although the last cygwin rebaseall I did seem to have helped a bit). (this is irrelevant to the current discussion, I'm just adding it here in case someone has the same issue later.)

R/

[Ryan Bis]

For what it's worth, I had tried using -j1, and it didn't seem to make any difference. The only thing that made it continue to the end was -f.

Also seems like canonical has changed their respository. I know I saw someone mention it before that they don't have the sun java version anymore. That was no big deal to install. Also, looks like the readline library was replaced with a different one (although apt-get tells you which package, so you can easily just change it).

I'm still getting errors in the sync, but hopefully it's nothing I need. Here's what has popped up so far:

error: RPC failed; result=6, HTTP code = 0
fatal: The remote end hung up unexpectedly
Exception in thread Thread-90:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 552, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 505, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/impinc/.repo/repo/subcmds/sync.py", line 200, in _FetchHelper
    clone_bundle=not opt.no_clone_bundle)
  File "/home/impinc/.repo/repo/project.py", line 934, in Sync_NetworkHalf
    and self._ApplyCloneBundle(initial=is_new, quiet=quiet):
  File "/home/impinc/.repo/repo/project.py", line 1475, in _ApplyCloneBundle
    exist_dst = self._FetchBundle(bundle_url, bundle_tmp, bundle_dst, quiet)
  File "/home/impinc/.repo/repo/project.py", line 1539, in _FetchBundle
    raise DownloadError('%s: %s ' % (req.get_host(), str(e)))
DownloadError: android.googlesource.com: <urlopen error [Errno -2] Name or service not known> 

Exception in thread Thread-91:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 552, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 505, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/impinc/.repo/repo/subcmds/sync.py", line 200, in _FetchHelper
    clone_bundle=not opt.no_clone_bundle)
  File "/home/impinc/.repo/repo/project.py", line 934, in Sync_NetworkHalf
    and self._ApplyCloneBundle(initial=is_new, quiet=quiet):
  File "/home/impinc/.repo/repo/project.py", line 1475, in _ApplyCloneBundle
    exist_dst = self._FetchBundle(bundle_url, bundle_tmp, bundle_dst, quiet)
  File "/home/impinc/.repo/repo/project.py", line 1539, in _FetchBundle
    raise DownloadError('%s: %s ' % (req.get_host(), str(e)))
DownloadError: android.googlesource.com: <urlopen error [Errno -2] Name or service not known> 

error: Couldn't resolve host 'android.googlesource.com' while accessing https://android.googlesource.com/platform/external/libnl-headers/info/refs

fatal: HTTP request failed
error: Cannot fetch platform/external/libnl-headers
warn: --force-broken, continuing to sync


It's still going right now, but figured I would give an update.


On Tuesday, April 3, 2012 5:06:08 PM UTC-4, Raphaël wrote:

[Ryan Bis]

Would the fact that this is running in a VM (VirtualBox) make any difference? I installed 64bit Ubuntu last night, and the sync still dies at different spots on me. It seems like the connection just drops all of a sudden. Then I have to kill the terminal, and restart repo sync again.

[Ryan Bis]

Ok, now when running createsymlinks, I get this:

target Dex: framework
make: *** [out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/noproguard.classes-with-local.dex] Killed

It takes a long time for that to error out (around 30 mins or so). Any ideas?

[Michael Wright]

Are you possibly running out of memory? "Killed" makes me think you're getting hit by the Linux OOM killer. Try checking dmesg or running `grep -i 'killed process' /var/log/kern.log`.

Michael

[Ryan Bis]

Yeah, I believe that's the problem. I allocated a little more RAM, and it got beyond that point. I guess I just don't have enough RAM to do this properly (it says on the site that you should have 16GB RAM/swap.). My whole system has 3, and I'm allocating 1.5GB to it now. I'll have to try to find some other solution (maybe try hacking my way around windows). It's just weird that I was able to get r15, and r16 to work with only 1GB of RAM.

On Wednesday, April 4, 2012 9:16:59 PM UTC-4, Michael Wright wrote:

Are you possibly running out of memory? "Killed" makes me think you're getting hit by the Linux OOM killer. Try checking dmesg or running `grep -i 'killed process' /var/log/kern.log`.

Michael

[Michael Wright]

If you have hard drive space to spare, just up the space allocated to your swap partition. 

Michael