SVN: PCSWMM Use

[Jason Kimmet]

Subversion Users, 

We are working on a large stormwater modeling program where we will be keeping track of a lot of PCSWMM file type models. We are exploring version management options and have come across the SVN community. I understand this is great for text-style files, however, I would like to know if you have seen users utilize this for engineering plans such as PCSWMM?

Thank you, 

Jason Kimmet Engineering Project Manager Associate Project Manager
.	.	T: 9018812985 M: 419-309-0922 jki...@allworldmail.com  www.allworldpm.com 60 N. B.B. King Blvd. Memphis, TN, 38103
		.
.

[David Chapman]

On 9/9/2021 1:53 PM, Jason Kimmet wrote:

Subversion Users, 

We are working on a large stormwater modeling program where we will be keeping track of a lot of PCSWMM file type models. We are exploring version management options and have come across the SVN community. I understand this is great for text-style files, however, I would like to know if you have seen users utilize this for engineering plans such as PCSWMM?

Many version control systems, Subversion included, work best when the new version of a file looks much the same as the previous version.  Otherwise you could be storing multiple full versions of the files.

Some features of Subversion (like "show lines that changed") aren't available for binary files, but if the changes in a binary file are limited to specific sections, you still get space-saving benefits in the repository; see http://help.collab.net/index.jsp?topic=/faq/svnbinary.html and https://svnbook.red-bean.com/en/1.7/svn.forcvs.binary-and-trans.html (both links are old, but binary file handling shouldn't be worse now than then).

A quick search didn't tell me anything about the PCSWMM format; how much of a file changes between model runs?  Is it text or binary?  Are the files very large (gigabytes or more)?  Are they inputs to software, or outputs?  Generally it is assumed that output files can be recreated given the full input configuration, so program outputs are often left out of the repository.
--

    David Chapman      dcch...@acm.org
    Chapman Consulting -- San Jose, CA
    EDA Software Developer, Expert Witness
    www.chapman-consulting-sj.com
    2018-2019 Chair, IEEE Consultants' Network of Silicon Valley

[Justin MASSIOT | Zentek]

Hi Jason!

David is right, I don't know PCSWMM either and you didn't provide any information about the file format used/generated by this software.

And I second what David says: the way to go is to "version control" the source files ; we don't care about output files much because generally we don't want to put them under version control, since we are able to re-generate all of them from the source files.

If your source files are in binary format, then you're in the same situation as mine: I work everyday with binary files of ~10Mo and Subversion really works well for that. If you're interested I wrote a blog article discussing the use of SVN or Git in the Electronic/Mechanical engineering context (CAD files).

Unfortunately, with binary files you will generally miss the "diff" feature which David talked about. Unless your software supports a "file comparison" feature! Example: Altium Designer (electronic CAD)

Even MS Office files can be diff'ed quite easily ;-)

Hope it helps.

Justin MASSIOT  |  Zentek

[Jason Kimmet]

David,

PCSWMM files normally won't exceed 1 gigabyte. Rest of answers: 

• how much of a file changes between model runs?  

Really depends … ranging from if perhaps a global model parameter edit is done during an iteration, or just changes to a study area like a detention pond.

• Are the files text or binary?  

Text

• Are they inputs to software, or outputs?  

Input file gets modified during design/analysis by the modeler.

Hopefully, this helps. thanks, 

 

Jason Kimmet
www.allworldpm.com

---

From: David Chapman <dcch...@acm.org>
Sent: Thursday, September 9, 2021 4:29 PM
To: Jason Kimmet <jki...@allworldmail.com>; us...@subversion.apache.org <us...@subversion.apache.org>
Subject: Re: SVN: PCSWMM Use

 

[David Chapman]

On 9/14/2021 9:26 AM, Jason Kimmet wrote:

(snipped to avoid top-posting)

You should be able to use Subversion.  My normal use case is for much smaller files (software source code) but all you need is disk space, and any version control software will have to deal with the differences between file versions.  I presume that users are not typing a gigabyte each time, and that the PCSWMM files are assembled by a tool from other data.  In theory you could store the inputs to whatever tool is generating a particular PCSWMM file version, but that may be too much of a change from your present workflow.

The fact that they are text files is good too.  I work in the semiconductor industry, and the binary files our tools generate (GDS and OASIS format, in case anyone cares) tend to have major differences between versions.  We'd end up storing the entire file each time, even if the files are functionally equivalent - in large part, the formats are order independent, and tool A's ordering convention tends to differ from Tool B's convention.  That is a problem when a chip-level GDS file can be a terabyte in size!

[David Chapman]

On 10/6/2021 8:43 AM, Jason Kimmet wrote:

(snipped to avoid top-posting)

---

From: David Chapman <dcch...@acm.org>
Sent: Tuesday, September 14, 2021 5:02 PM

David, 

Thank you for that information it's very helpful - What I am worried about is lacking the appropriate expertise to translate the input changes from the models. We are a team of civil/water resource engineers and wouldn't be able to decipher the backend code of the models, meaning the subversion would ned to be able to open as more of a frontend/visual model representation that appears in the PCSWMM software (see below for the example of the visual output example). 

is this something that is offered in the subversion?

thank for you help!

 

Jason Kimmet
www.allworldpm.com

Basically, Subversion will store all versions of your model files and directories in the repository, then provide you a way to check out  a working copy.  This working copy may come from the latest versions of all files (the default) or from some other state of the files.  The user may ask for a working copy that contains all directories or a subset.

Your existing tools would then operate in the working copy, as if it was an ordinary directory containing ordinary files.  If a tool modifies a file and you want to keep the changes, you would commit the new version to the repository.  This is standard operating procedure for version control systems, and you need to understand how this works before beginning to use one.

Subversion, like all version control systems, doesn't know anything about how to display or modify your files.

Subversion is free, and it's easy to set up an experimental Subversion repository on a local machine while you learn the concepts.  This makes Subversion simple for a single person to try out.  Of course, you'd still want a network-savvy IT person to configure and administer (backups, backups, backups!) a network-accessible repository.

[Jason Kimmet]

What do you recommend as the easiest subversion for experimental purposes?

thanks, 

 

Jason Kimmet
www.allworldpm.com

---

From: David Chapman <dcch...@acm.org>
Sent: Wednesday, October 6, 2021 12:06 PM

[Mark Phippard]

On Wed, Oct 6, 2021 at 1:26 PM Jason Kimmet <jki...@allworldmail.com> wrote:

What do you recommend as the easiest subversion for experimental purposes?

If you are on Windows, install TortoiseSVN. Create a repository somewhere on your local disk and access it via file:// protocol. If all goes well then you just need to install a server somewhere that you probably would access via https://

If you have never used SVN then the first 2 chapters of the SVN Book are highly recommended as an introduction.

https://svnbook.red-bean.com/en/1.7/index.html

The book uses the command line client for everything as opposed to the TortoiseSVN GUI, but that is the best way to learn. TSVN can install the command line client as part of its installer.

If you are on MacOS os Linux, then just install the command line client and try that ... again using a local repository and file:// protocol.

Mark

 

[Jason Kimmet]

David, 

Thank you for that information it's very helpful - What I am worried about is lacking the appropriate expertise to translate the input changes from the models. We are a team of civil/water resource engineers and wouldn't be able to decipher the backend code of the models, meaning the subversion would ned to be able to open as more of a frontend/visual model representation that appears in the PCSWMM software (see below for the example of the visual output example). 

is this something that is offered in the subversion?

thank for you help!

 

Jason Kimmet
www.allworldpm.com

---

From: David Chapman <dcch...@acm.org>
Sent: Tuesday, September 14, 2021 5:02 PM

[Lorenz]

Mark Phippard wrote:

>On Wed, Oct 6, 2021 at 1:26 PM Jason Kimmet <jki...@allworldmail.com>
>wrote:
>
>> What do you recommend as the easiest subversion for experimental purposes?
>>
>>
>If you are on Windows, install TortoiseSVN. Create a repository somewhere
>on your local disk and access it via file:// protocol. If all goes well
>then you just need to install a server somewhere that you probably would
>access via https://
>
>If you have never used SVN then the first 2 chapters of the SVN Book are
>highly recommended as an introduction.
>
>https://svnbook.red-bean.com/en/1.7/index.html

Tortoise SVN comes with a very good online help.
The first chapters are very similar to the SVN book.

>The book uses the command line client for everything as opposed to the
>TortoiseSVN GUI, but that is the best way to learn. TSVN can install the
>command line client as part of its installer.
>
>If you are on MacOS os Linux, then just install the command line client and
>try that ... again using a local repository and file:// protocol.
>
>Mark

--

Lorenz