Thought on Releasing Joshua 6.0.5

[Lewis John Mcgibbney]

Hi Folks,

What are the thoughts on releasing 6.0.5?

I am working on the homebrew recipe and bumping in to some nasty problems with the 6.0.4 which is preventing the Formula from being merged. Examples include

• artifact naming; can we please change the artifact to 6.0.5 instead of v6.0.5
• excess files such as
  
• The offending files are:
    /usr/local/Cellar/joshua-v/6.0.4/lib/eng_sm6.gr
    /usr/local/Cellar/joshua-v/6.0.4/lib/hadoop-0.20.2.tar.gz
    /usr/local/Cellar/joshua-v/6.0.4/lib/README
   * Non-executables were installed to "/usr/local/Cellar/joshua-v/6.0.4/bin"
  The offending files are:
    /usr/local/Cellar/joshua-v/6.0.4/bin/.DS_Store
    /usr/local/Cellar/joshua-v/6.0.4/bin/.gitignore
  

The Formula I've written [0] can also be run with the --HEAD flag. When I do this all the homebrew auditing, testing, etc. is compliant. If we were to release Joshua master it would mean that Formula could be updated and the brew recipe could get merged.

What do you guys think?

Thanks

[0] https://github.com/Homebrew/homebrew/pull/44490/files

--

Lewis

[Matt Post]

Hi,

I'll release all of this as soon as the eclipse stuff is ironed out.

Regarding the problems below:

- Yes, I'll use 6.0.5, dropping the v

- hadoop 0.20.2 can be removed

- eng_sm6.gr should be retained; it is the model file for the (monolingual) parser, which is used for parsing the target side of parallel corpora. HOWEVER, it might be preferable to clone the berkeley parser (now on github, I see), and download that when needed. This would also allow removing lib/BerkeleyParser.jar from the repo.

- Is lib/README really a problem?

- bin/.DS_Store is not in the repo

- bin/.gitignore can be removed if its contents are added to the top-level .gitignore (and prepended with bin/ of course)

matt

--
You received this message because you are subscribed to the Google Groups "Joshua Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to joshua_develop...@googlegroups.com.
To post to this group, send email to joshua_d...@googlegroups.com.
Visit this group at http://groups.google.com/group/joshua_developers.
For more options, visit https://groups.google.com/d/optout.

[Lewis John Mcgibbney]

Hi Matt,
No problems, I'll have a look at this ASAP.
Thanks
Lewis

[Lewis John Mcgibbney]

Hi Matt,

On Thursday, 8 October 2015 05:01:41 UTC-7, Matt Post wrote:

Hi,

I'll release all of this as soon as the eclipse stuff is ironed out.

So what is it with the Eclipse stuff that is not working? The tickets are closed so I am assuming everything is working. 
Thanks

[Matt Post]

Yes, I believe the eclipse piece is fixed. However, there is still the hanging issue about telling ant-eclipse to exclude src/kenlm from the class path.

[Lewis John Mcgibbney]

Hi Matt,
Please see
https://github.com/joshua-decoder/joshua/pull/231
It is annoying. My opinion is that this is not a blocker to a release.
Thanks

[Matt Post]

Agreed, I don't think it will hold up, I hope to do the release build today.

<Screen Shot 2015-10-21 at 8.56.16 PM.png>

[Lewis John Mcgibbney]

Dynamite Matt.

I'll progress with the Homebrew recipie once 6.0.6 is out.

Then comes completing languages models and full blown Ingegration into Apache Tika's translate API

You received this message because you are subscribed to a topic in the Google Groups "Joshua Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/joshua_developers/NntKkwwx2B0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to joshua_develop...@googlegroups.com.

To post to this group, send email to joshua_d...@googlegroups.com.
Visit this group at http://groups.google.com/group/joshua_developers.
For more options, visit https://groups.google.com/d/optout.

--
Lewis

[Lewis John Mcgibbney]

Hey Matt,

Any updates with the release? Were there some problems?

Thanks

Lewis

--

Lewis

[Matt Post]

I'm a little behind with some changes pushed up from Amazon, but am nearly finished testing. New date target is November 3. Sorry for the continual delays (meant to email you today).

[Lewis John Mcgibbney]

NP at all Matt. We've all been there.

Thanks for update. I'll shift my time here to development of the Russian --> English language pack.

Thanks

[Matt Post]

Hi Lewis,

I've build the 6.0.5 release and will update the web page and announce it later tonight.

One issue: the tarball is 285 MB, due in part to the inclusion of libraries and other tarballs that are only need if you want to *build* models (mostly hadoop, which is ~150 MB). For the language packs, we could put together a version that only includes the main decoder and decode-time dependencies, and it'd be a lot smaller. I still want to maintain the full release for people who do want to build, but

(a) do you think a decoder-only version would be useful for brew?

(b) what should I name it?

matt

[Lewis John Mcgibbney]

Hi Matt,

EXCELLENT :)

On Wed, Nov 4, 2015 at 2:03 PM, Matt Post <po...@cs.jhu.edu> wrote:

Hi Lewis,

I've build the 6.0.5 release and will update the web page and announce it later tonight.

Nice

 

One issue: the tarball is 285 MB, due in part to the inclusion of libraries and other tarballs that are only need if you want to *build* models (mostly hadoop, which is ~150 MB). For the language packs, we could put together a version that only includes the main decoder and decode-time dependencies, and it'd be a lot smaller.

Yes I understand. It is quite chunky... but I think that it is valuable for those that wish to build models... which will hopefully increase due to ongoing activity.

 

I still want to maintain the full release for people who do want to build, but

(a) do you think a decoder-only version would be useful for brew?

Yes I think it would. The decoder itself can be used for testing and validation, etc. It is always good to have the artifacts out there. If we find that it is not useful then we can go back a later stage however it is always good to have it. I can add a flag the the Homebrew formula to define which artifact one installs.

 

(b) what should I name it?

Whatever you name it please make the version separate from the text e.g. joshua-6.0.5.tar.gz (for decoder only) and joshua-full-6.0.5.tar.gz for the decoder only artifact.

Hows does that sound?

Thanks and great work.

lewis

[Matt Post]

> Whatever you name it please make the version separate from the text e.g. joshua-6.0.5.tar.gz (for decoder only) and joshua-full-6.0.5.tar.gz for the decoder only artifact.
> Hows does that sound?
> Thanks and great work.
> lewis

I think I will do joshua-6.0.5.tgz as the full system, since that's been the default, and joshua-runtime-6.0.5.tgz as the light version. This way it's clear to people what they are getting.

The former has been pushed up and the website updated. I'll add the runtime version tomorrow.

matt

[Lewis John Mcgibbney]

Saw that. I Favorited and reposted the tweets.

Pushing the Homebrew formula right now.

I'll update Github in due course.

Ta

matt

--
You received this message because you are subscribed to a topic in the Google Groups "Joshua Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/joshua_developers/NntKkwwx2B0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to joshua_develop...@googlegroups.com.
To post to this group, send email to joshua_d...@googlegroups.com.
Visit this group at http://groups.google.com/group/joshua_developers.
For more options, visit https://groups.google.com/d/optout.

--

Lewis