Sync rate testing and sync tool version

[David Chandek-Stark]

I was just trying out the sync rate testing tool.  It seems to work fine, but I noticed it's using version 3.1.1 of the synctool.  I'm intending to use v4.1.0 of the synctool.  Just wondering if the sync rate tester needs to be updated.

Thanks,

David

[Bill Branan]

Hi David,

Your note helped me to realize that I hadn't added a comment at the top of the SyncRate documentation page to point to a more current iteration of that tool, which we now call SyncOptimize. The SyncOptimize tool is specifically focused on selecting the thread count number which will provide the greatest throughput when using the SyncTool. It does this really simply by running the SyncTool over a set of files (you specify how many and how large, so you can tailor it to your data set) several times, varying the thread count and eventually narrowing down to the count which provides the best transfer time.

The documentation for SyncOptimize is here: https://wiki.duraspace.org/display/DURACLOUDDOC/DuraCloud+SyncOptimize+Tool. It's part of the primary baseline, so it gets updated with each release. It's also built in to the UI version of the SyncTool. If you are using the UI, just go to the bottom of the configuration tab and you'll see the option to perform an optimize run. Of course, you have more options when you run it via the command line, and you get to see all of the output as well.

best,

Bill

--
You received this message because you are subscribed to the Google Groups "DuraCloud Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to duracloud-users+unsubscribe@googlegroups.com.
To post to this group, send email to duracloud-users@googlegroups.com.
Visit this group at https://groups.google.com/group/duracloud-users.
For more options, visit https://groups.google.com/d/optout.

[David Chandek-Stark]

Thanks, Bill.  I tried an initial run of SyncOptimize and got this error:

Exception in thread "main" java.lang.RuntimeException: Unable to create ContentStore due toError retrieving content stores. Illegal character in authority at ind/durastore/storese.duracloud.org

at org.duracloud.syncoptimize.test.SyncTestManager.getContentStore(SyncTestManager.java:151)

at org.duracloud.syncoptimize.test.SyncTestManager.<init>(SyncTestManager.java:55)

at org.duracloud.syncoptimize.SyncOptimizeDriver.getOptimalThreads(SyncOptimizeDriver.java:88)

at org.duracloud.syncoptimize.SyncOptimizeDriver.main(SyncOptimizeDriver.java:132)

On Friday, August 12, 2016 at 10:51:10 AM UTC-4, Bill Branan wrote:

Hi David,

Your note helped me to realize that I hadn't added a comment at the top of the SyncRate documentation page to point to a more current iteration of that tool, which we now call SyncOptimize. The SyncOptimize tool is specifically focused on selecting the thread count number which will provide the greatest throughput when using the SyncTool. It does this really simply by running the SyncTool over a set of files (you specify how many and how large, so you can tailor it to your data set) several times, varying the thread count and eventually narrowing down to the count which provides the best transfer time.

The documentation for SyncOptimize is here: https://wiki.duraspace.org/display/DURACLOUDDOC/DuraCloud+SyncOptimize+Tool. It's part of the primary baseline, so it gets updated with each release. It's also built in to the UI version of the SyncTool. If you are using the UI, just go to the bottom of the configuration tab and you'll see the option to perform an optimize run. Of course, you have more options when you run it via the command line, and you get to see all of the output as well.

best,

Bill

On Fri, Aug 12, 2016 at 10:33 AM, David Chandek-Stark <dchand...@gmail.com> wrote:

I was just trying out the sync rate testing tool.  It seems to work fine, but I noticed it's using version 3.1.1 of the synctool.  I'm intending to use v4.1.0 of the synctool.  Just wondering if the sync rate tester needs to be updated.

Thanks,

David

--
You received this message because you are subscribed to the Google Groups "DuraCloud Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to duracloud-use...@googlegroups.com.
To post to this group, send email to duraclo...@googlegroups.com.

[Bill Branan]

Hi David,

Sorry to hear it's not working. From the error you posted it sounds like the tool may be having trouble with a special character in your password. I know we've run into issues of that type before, as noted here: https://jira.duraspace.org/browse/DURACLOUD-969. We found in that case that we just needed to escape characters on the command line in order for the password to be passed correctly from the command line into the Java app. So maybe start by escaping the special characters in your password.

If that doesn't work, you can verify that this is a password character issue by changing your password temporarily to an alphaNumeric value (Go to https://manage.duracloud.org, log in, select My Profile in the upper right corner) and see if the optimize tool will run at that point. If that's the case, please either open a ticket in Jira or let me know so I can open a ticket. In either case, it would be helpful to know the character or characters which seem to be causing the problem.

Of course, once you're done, you can always change your password back.

best,

Bill

To unsubscribe from this group and stop receiving emails from it, send an email to duracloud-users+unsubscribe@googlegroups.com.
To post to this group, send email to duracloud-users@googlegroups.com.

[Bill Branan]

Hi David,

I'm glad to hear the mystery is solved. I'm replying back on-list just to ensure that the resolution is captured there for anyone having this problem in the future.

Bill

On Tue, Aug 23, 2016 at 1:56 PM, David Chandek-Stark <dchand...@gmail.com> wrote:

Bill,

I solved the problem.  Somehow (no idea how) I was using a file with DOS line terminators to export env variables like DURACLOUD_HOST.  Ran dos2unix on it and all is well.

:(

--David