running the buildserver on Windows

[Gary Frederick]

close

start-time Feb 4, 2012 6:28:21 PM CST
uptime-in-ms 44832
vm-name Java HotSpot(TM) Client VM
vm-vender Sun Microsystems Inc.
vm-version 20.5-b03
os-arch x86
os-name Windows 7
os-version 6.1
num-processors 8
load-average-past-1-min -1.0
total-memory 16252928
free-memory 7997000
max-memory 1490026496
used-heap 8255928
used-non-heap 11245712
count-async-build-requests 0
rejected-async-build-requests 0
maximum-simultaneous-build-tasks unlimited
completed-build-tasks 0
active-build-tasks 0 

I took the source Kim was working with and did some moving around and

  still not done but so close!!!

Gary

[M. Hossein Amerkashi]

Gary,

The problem will be in packaging of aps and saving into local pc. I can connect to my buildserver[1], but get A LOT OF path issue[2]. Also, I'm also getting "unable to create jvm" error

[1]
start-time Feb 5, 2012 12:11:17 AM EST
uptime-in-ms 17605

vm-name Java HotSpot(TM) Client VM
vm-vender Sun Microsystems Inc.

vm-version 20.0-b11

os-arch x86
os-name Windows 7
os-version 6.1

num-processors 2
load-average-past-1-min -1.0
total-memory 16252928
free-memory 8127376
max-memory 259522560
used-heap 8125552
used-non-heap 20150664

count-async-build-requests 0
rejected-async-build-requests 0
maximum-simultaneous-build-tasks unlimited
completed-build-tasks 0
active-build-tasks 0

[2]
https://groups.google.com/d/msg/app-inventor-open-source-dev/yNxipgw3rrk/9CFAFunix-YJ

[Gary Frederick]

Some more notes

Kim changed places where "/" was hard coded in files!!!

I'm using File.separator.

I'm building on Windows 7 64 bit with Java 6 32 bit from Eclipse. I wanted to use 32 bit Java on the theory that most have it on Windows. Because of the problem with running out of heap space with 32bit Java, I set both buildserver.Main and buildserver.BuildServer to 1470.

  int childProcessRamMb = 1470;

It ran out of memory trying to run DX with it set to 2048, and was happy (if a bit slow) with 1470. I may play with it

  or I may abandon my idea of using a 32bit Java for a 64bit Java.

I got it up and the unit tests mostly ran (over 85% passed), the problems are more "/" vs "\" issues.

I am looking at the code in Compiler.getResource and related areas. If I use this

  public static final String RUNTIME_FILES_DIR = File.separator + "files" + File.separator;

it dies, but the constant from the source I got works

  public static final String RUNTIME_FILES_DIR = "/files/";

The unit tests that failed are in the buildserver's YailEvalTest and one in blockseditor jsonp UtilTest

I had this really smart ass comment about the files used to test components...util.MediaUtilTest

But am tooo busy listening to 17 Candle - Follow Me Down and am about to listen to Zack Borer- "That's The Way"

I commented all the code in that test out, till I get the file names under control.

I am trying to decide if there is a Windows gotcha in Compiler.getResource here

  file = File.createTempFile(prefix, suffix);

or here

  Files.copy(Resources.newInputStreamSupplier(Compiler.class.getResource(resourcePath)),

       file);

I am thinking of putting another vm with Ubuntu, Java 6 64 bit and Eclipse to have something that works

  or I may just keep pounding my head against the windows wall :-)

Once the buildserver is running on Windows, I want to see if we can generate an .apk that can be installed in the Market

  and look at a lot of other things.

I like the code, Google did very good!!!

I like that it is so easy to work with in Eclipse. I do need to convince it that it is a Java project, but that can wait till it is up

  or I get frustrated with not having more good support from Eclipse. Ant is nice.

I am using Mercurial so it should be easy to get a diff and to merge the next set of changes we get from MIT.

Woo Hoo

Gary

jsoft on G+

[Student Kim]

This is the way how to run ant build on windows.

https://docs.google.com/document/d/1Cp7Lpp7QjhNEkU2osLUof6lq-FoqCKU0Ir_mekdBP2M/edit?hl=en_US

[Gary Frederick]

I added a few comments.

I'm going to go poke at the kawa part a bit

Summary

Tests	Failures	Errors	Success rate	Time
442	0	72	83.71%	80.372

Note: failures are anticipated and checked for with assertions while errors are unanticipated.

Packages

Name	Tests	Errors	Failures	Time(s)	Time Stamp	Host
com.google.appinventor.blockseditor.jsonp	14	1	0	0.096	2012-02-06T09:03:12	bs
com.google.appinventor.blockseditor.youngandroid	11	0	0	0.043	2012-02-06T09:03:14	bs
com.google.appinventor.buildserver	73	71	0	0.369	2012-02-06T09:04:25	bs

and more...

the buildserver errors are all have the same problem, they can not load C:UsersGaryAppDataLocalTempsimple_components1015487638226342634.scm
which looks like an issue with file separators.

YailEvalTest

testBasicKawaEval

Error

cannot load C:UsersGaryAppDataLocalTempsimple_components1015487638226342634.scm (The system cannot find the file specified) java.lang.RuntimeException: cannot load C:UsersGaryAppDataLocalTempsimple_components1015487638226342634.scm (The system cannot find the file specified) at kawa.standard.load.apply2(load.java:47) at kawa.standard.load.apply1(load.java:26) at gnu.mapping.Procedure.apply(Procedure.java:130) at gnu.mapping.Procedure.apply(Procedure.java:113) at gnu.mapping.CallContext.runUntilDone(CallContext.java:234) at gnu.expr.ModuleExp.evalModule2(ModuleExp.java:383) at gnu.expr.ModuleExp.evalModule(ModuleExp.java:201) at gnu.expr.Language.eval(Language.java:943) at gnu.expr.Language.eval(Language.java:883) at gnu.expr.Language.eval(Language.java:865) at com.google.appinventor.buildserver.YailEvalTest.setUp(YailEvalTest.java:49)

0.1

[Gary Frederick]

and

in the buildserver test source YailEvalTest

cruely hack it with this

  private static final String myYailRuntimeLibrary = "C:/Users/ai/AppData/Local/runtime.scm";

  private static final String myYailEvalTest = "C:/Users/ai/AppData/Local/YailEvalTest.scm";

...

      scheme.eval("(load \"" + myYailRuntimeLibrary + "\")");

      scheme.eval("(load \"" + myYailEvalTest + "\")");

and put the two files in your local file system

and all the unit tests pass

there are still two failures, more '\' vs '/' things

but

!!!

[Gary Frederick]

note!!!

putting the two scm files in a known location is a HACK

but it lets me move forward

Gary

jsoft on G+

[Gary Frederick]

and after another long day

buildserver's source file Compiler.java has this

Files.copy(Resources.newInputStreamSupplier(Compiler.class.getResource(resourcePath)),

            file);

and as we all know (now...)

getResource protects the name IF it starts with a '/'

http://docs.oracle.com/javase/6/docs/api/java/lang/Class.html#getResource(java.lang.String) 

I did serious hackage to find that

1. I am going to take a break
   
2. build the buildserver zip
   
3. test (it was soooo close before)
   

and there may be a LONG time between step 1 and step 2 :-)

Gary

[Tu App Inventor]

look http://www.tuappinvetorandroid.com/servidores-p%C3%BAblicos-de-tu-app-inventor/

On 6 feb, 22:20, Gary Frederick <g...@jsoft.com> wrote:
> and after another long day
>
> buildserver's source file Compiler.java has this
>
> Files.copy(Resources.newInputStreamSupplier(Compiler.class.getResource(reso urcePath)),
>             file);
>
> and as we all know (now...)
>

> getResource protects the name IF it starts with a '/'http://docs.oracle.com/javase/6/docs/api/java/lang/Class.html#getReso...)

>
> I did serious hackage to find that
>

>    1. I am going to take a break
>    2. build the buildserver zip
>    3. test (it was soooo close before)

[Jesse Sharps]

I'm not sure which file separators you have and haven't replaced yet, but you can also pass those junit tests by replacing the return in getResource with
return file.getAbsolutePath().replace("\\", "/");

I take it you added in the aapt.exe from the Windows Android SDK and pointed compiler.java at it already?

Also, I think Execution.java needs some help in the execute method right at the beginning...
for(int i =0; i < command.length; i++){
          command[i] = command[i].replace("\"", "\\\"");
}

I've been getting rejected in the kawa compile stage, which is interesting considering I can build the buildserver on Windows and as part of that it packages a sample app.  Keeps complaining about an invalid something or other in the scm file.  Been a while since I last looked at getting it up and running.
Hopefully something here was new to you and helps you out.

[Gary Frederick]

Howdy Jesse

I almost missed your good comment,

I am reading and writing this topic in the user supported getting set up and connecting with App Inventor - Google Groups

   https://groups.google.com/forum/?hl=en#!forum/usgsuacwai 

I'll follow up in a bit, almost caught up

Gary

[Gary Frederick]

I am using an axe. I like your two suggestions, especially after I finally realized what BuildServer.class.getResource was doing.

I'll add your code and back out some of the damage I did with the axe.

On Wednesday, February 8, 2012 5:04:47 AM UTC-6, Gary Frederick wrote:

Howdy Jesse

I almost missed your good comment,

I am reading and writing this topic in the user supported getting set up and connecting with App Inventor - Google Groups

   https://groups.google.com/forum/?hl=en#!forum/usgsuacwai 

I'll follow up in a bit, almost caught up

Gary

On Wednesday, February 8, 2012 1:16:39 AM UTC-6, Jesse Sharps wrote:

I'm not sure which file separators you have and haven't replaced yet, but you can also pass those junit tests by replacing the return in getResource with
return file.getAbsolutePath().replace("\\", "/");

I take it you added in the aapt.exe from the Windows Android SDK and pointed compiler.java at it already?

yes

 

Also, I think Execution.java needs some help in the execute method right at the beginning...
for(int i =0; i < command.length; i++){
          command[i] = command[i].replace("\"", "\\\"");
}

I've been getting rejected in the kawa compile stage, which is interesting considering I can build the buildserver on Windows and as part of that it packages a sample app.  Keeps complaining about an invalid something or other in the scm file.  Been a while since I last looked at getting it up and running.

from a build

...

Feb 6, 2012 10:43:39 AM com.google.appinventor.buildserver.Execution execute

INFO: ____Executing C:\Program Files (x86)\Java\jdk1.6.0_30\jre\bin\java -mx1270M -cp <big trim>\simple_components4777654889271457989.scm

Feb 6, 2012 10:43:40 AM com.google.appinventor.buildserver.Compiler generateClasses

INFO: Kawa compile time: 0.561 seconds

Feb 6, 2012 10:43:40 AM com.google.appinventor.buildserver.Compiler generateClasses

INFO: Can't find class file: <moderate trim>\test\Screen1.class

<br>appinventor\ai_ggf\test\Screen1.yail:5:1: un-implemented expression in modul<br>Kawa compile time: 0.561 seconds

Build error output: Error: Your build failed due to an error when compiling 

I can not think of a way to apply an axe so I may try your replace :-)

 

Hopefully something here was new to you and helps you out.

Axe vs what should be done...

Thanks!

Gary

[김진우]

Hi Jesse Sharps :) First thanks for commenting the bug - that was the
problem I was stuck.
So the solution is updating right now.

If you get these solution, it works properly to the apk packaging
steps.
It gives me apk file, but there's some problem with DX or something so
when we install and run,
it gives me an error.

If this problems solved - then my docs would be version beta.
But anyway,
https://docs.google.com/document/d/1Cp7Lpp7QjhNEkU2osLUof6lq-FoqCKU0Ir_mekdBP2M/edit?hl=en_US
here you go.

[Kyle Hamilton]

https://code.google.com/r/aerowolf-appinventor has all of the code
changes necessary to get BuildServer running and correctly compiling
the SimpleTestApp on Windows. It also has the changes necessary to
get the YailEvaluationTests working. Among others, there's changes to
appserver/build.xml (to copy aapt.exe as well as aapt), Compiler.java,
BuildServer.java, ProjectBuilder.java...

64-bit Java is probably the win here.

-Kyle H

> --
> You received this message because you are subscribed to the Google Groups "App Inventor Open Source Development" group.
> To post to this group, send email to app-inventor-o...@googlegroups.com.
> To unsubscribe from this group, send email to app-inventor-open-so...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/app-inventor-open-source-dev?hl=en.
>

[김진우]

Good idea of making project in google project hosting

But will it make apk file when we press download to computer?

You know eventhough when you succeed with packaging simple testapp
apk

It might not really work when we press download to computer.

[Gary Frederick]

Houston, we have apk!

I downloaded an app with the buildserver running on Windows

and it runs.

I am going to splash water on my face

and clean up my code

  and document

The final key was using Jesse's ... .replace("\\", "/")

  https://groups.google.com/d/msg/app-inventor-open-source-dev/wuHKMFZoQvc/U0A7YCEKEugJ

in YABlockCompiler's packageNameFromPath

Woo Hoo

Gary

[Gary Frederick]

start-time Feb 11, 2012 4:14:42 AM CST

uptime-in-ms 1702085

vm-name Java HotSpot(TM) Client VM

vm-vender Sun Microsystems Inc.

vm-version 20.5-b03

os-arch x86

os-name Windows 7

os-version 6.1

num-processors 8

load-average-past-1-min -1.0

total-memory 23097344

free-memory 9906560

max-memory 1490026496

used-heap 13190784

used-non-heap 13101784

count-async-build-requests 4

rejected-async-build-requests 0

maximum-simultaneous-build-tasks unlimited

completed-build-tasks 4

active-build-tasks 0

woo hoo

It is going to take a while to get my code clean.

It is fast!!!

The DX step is the slowest part, but even there it is < 12 seconds.

The buildserver is visible on the internet. When I have things cleaned up, I will try running in developer mode.

I will get a 64 bit Java compiler up and see if the 64 bit version is faster. Note: faster != better.

Gary

[Gary Frederick]

and continuing to ignore what I said I would do, I tested and it does build and run with the personal server.

Hmmm

[Gary Frederick]

I have made progress, a personal App Inventor talks to a local buildserver nicely, it builds the .apk and is looking good.

I spent some time working on how to run it. I have it running on a USB, good news

it only works if you are running  Windows 7

  and your user name is Gary

  and you have the jdk where I put mine

  and and and...

:-)

I'm stepping back and getting something up, then we can work on various ways to package. I'll add a parameter to specify where the jarsigner is located and do the things Mark Friedman @ Google suggested here

  https://groups.google.com/d/topic/app-inventor-open-source-dev/292hsNN1kn0/discussion

and Hal @ MIT suggested here

  https://groups.google.com/d/topic/mit-appinventor-jars/6o5OmGHA9qA/discussion

I cloned a local copy of the 61 version from http://code.google.com/p/app-inventor-releases/ and added the changes without my hacks. I'm currently thinking of putting it into the http://code.google.com/p/ai-docs/ code repository. That should make it easy to merge back to the source from Google/MIT if they choose and for others to get if they are interested in the source.

I hope to have something out that I am proud of in the next day or two. It's up, now I need to make it work... for all.

Gary

[Roger Belk]

Did I miss a thread somewhere? You have always been Gary right?

Thank You
Roger Belk
Working from my NEXUS
www.bigdaddyapp.com
(903) 309-8465

On Feb 12, 2012 8:14 AM, "Gary" <taifu...@gmail.com> wrote:

ok, this one was easy, my new name is Gary now :-)

and I also have Windows 7...

and I have the jdk...

good work Gary!

the new Gary :-)

[Gary Frederick]

Our Linux server went down (interesting timing) yesterday. I switched to the backup buildserver runnng on Amazon EC2... and found that while it was ok doing a build here and there, it did not handle more that a few builds. When lemons are falling all over, make lemonade. We are currently running our company buildserver on Windows! 

I am working as hard as I can on getting things organized and available for all. Things work but there will be issues. 

Setting up the current configuration is not smooth. There are bound to be bugs running the buildserver on Windows. Folks will need more support.

I had a LOT of time to think about our goals as I compiled over and over and over.

 

One of our goals is to have configurations that are easy to set up and run. For example, I have a configuration that runs on a USB. Several schools are interested. They would be able to give students a USB and the students would be set up and connected on any PC. That is the theory and it is not far off. The buildserver running on Windows gets us a lot closer to that goal.

Another goal is to have a configuration of App Inventor that runs locally. App Inventor runs in the cloud. It also can run locally without internet. That is important for folks that do not have internet all the time or those that have expensive internet.

Turns out that both goals are solved with the same configuration.

It also gives hope for schools that provide more safety for their students or have firewalls that cause problems.

and who knows what else folks will come up with :-)

My company has a store. We sell support for schools and others getting their private servers up. We also offer accounts on our public server. We decided to change things a bit. Now we are providing support for $10 a month. The $10 includes an account on our public server and access to the configurations we are building. The servers and configurations are part of how we help folks.

I'll still help folks here. It is part of our goal of supporting App Inventor for All.

I'm going to put the source where others can get up and running. Getting the source ready to release is going to take time. I got much of the source cleaned up and then ran into a problem. I hacked the code in to add a command line parameter that has the location of the jarsigner. I am going to get the next version of the source, add our changes back in and hope to release it soon after MIT updates.

My goal is that this group continue to be a place we can all go for user supported getting set up and connecting with App Inventor issues. The source will probably be in the ai4a Google code site, ai-docs. The folks that have accounts on Jefferson Software's public server are using the Windows buildserver and they can get the zip files to run on their own Windows PC. That lets us get things running smoother before we release the source.

or we will do something else :-)

Gary

jsoft on G+

[McHanon]

And the conclution for an ordinary windows7 user:

Is the App Inventor available and ready for use ?

And where ?

[Gary Frederick]

I followed up in your post in the jars group.

Gary

user supported getting set up and connecting with App Inventor - Google Groups

[Gary Frederick]

It was from another group and had no good context.

The buildserver is working very well. I have had a grand total of one person get the Personal server that runs locally on Windows and it is not quite up, but I have hope :-)

and

I am working on another example of using JSON as I exercise the buildserver on Windows. It is pretty nifty. Pictures soon.

Gary

[Gary Frederick]

fwiw, the tAIR chat yesterday was a good one. 

  https://groups.google.com/d/topic/ai4a/80XHm5bkn5A/discussion

We talked of how to advertise apps and about generating Java from App Inventor in addition to talk about App Inventor on Windows. Several folks said they were getting it, and so far a total of zero (0) have followed up?

I thought there was a lot of interest?

  hmmm

One of my schools does not have their buildserver up and have a Windows server. I'll see if they want to test.

Gary