.gcode versus .s3g

[Eureka]

What is the difference between the file formats .gcode and .s3g? What exactly is replicatorG doing when it converts the gcode to a .s3g file? It takes some time, so it seems there has to be some calculations involved in the convertion process, yes?

Im really curious about this...

[Charlie Prevost]

I'm not an expert on this but, as far as I know, gcode is the instructions created by skeinforge (or whatever slicer) These need to be translated before sent to the bot as they are not something that it can work with directly. s3g are the instructions that the bot uses to control the extruder, HBP etc. When printing from the computer over USB, each instruction in gcode is translated and then sent. printing from SDcard needs all of the translations done in advance.
in summary: gcode is a product of the slicing up of a model, and are not machine specific (I don't think). It'll give a toolpath and other parameters.
S3g is specific instructions for a printer (e.g. replicator 1) created from the instructions in the gcode.

[Jetguy]

S3G is binary gcode instructions, where gcode is human readable. Also
note that in the conversion to S3G, your specific machine type
settings are used (steps per mm is the BIG one). It's a one way
conversion-gocde to S3G in that you cannot reconvert or read an S3G.
Thus, that S3G only works for the bot it was created for. Gocde is
genrally the instructions for the tool path. You'll note we typically
append a start and end gcode to the actual object printing part that
is in the middle. The middle part is what is created by the slicling
software and is not so much machine specific in theory, but more to
the exact nozzle and plastic type.

This is why you can use different slicers that create the printing
gcode based on nozzle size and filament diameter, then put that text
gcode into a printer console application, that uses the machine
"driver" with steps per mm and other functions to compile the text
gcode and send it to the bot, either in the S3G format or across the
USB. If you ever learn anything aboout programming, one of the most
CPU and time intensive things for a microcontroller is reading text
(ascii) across a serial connection. This is the reason we use binary
serial streams, rather than just sending the text gcode. It's always
made sense to use the horsepower of the host computer to do the ehavy
lifting and let the 8 bit microcontroller just run the instructions
directly in a format that is faster.

This is the HUGE why, you should always print from SD card. With all
the adavanced acceleration functions these days, the microcontroller
needs to read ahead and thus even faster serial communications are
happening (remember the AVR microcontrollers we use are serial only
and to do that, we have a USB to serial conversion since the serial
port has gone away. Printing from SD card lets the micronctroller not
waste time reading and buffering from the serial port, and read vlovk
by block from memory on the SD card. The result is very visible
prints. Tiny pauses can and often do happpen when printign from USB
and when that happens, the speed change in the nozzle moving leaves
little bumps on the surface of the print. Some computers are worse
than others, thus why many people ignore this tip, but the facts are
plain, there is a performance hit for printing from USB.

[ddurant]

> If you ever learn anything aboout programming, one of the most
> CPU and time intensive things for a microcontroller is reading text
> (ascii) across a serial connection. This is the reason we use binary
> serial streams, rather than just sending the text gcode.

 

I'm not sure that's entirely correct. Bytes is bytes, ASCII or not..

 

Also, it's definitely possible for a binary protocol to end up taking more bytes than a text one. Dunno how efficient s3g is but gcode text is pretty efficient, allowing values that haven't changed to be left out; Z and F for example, frequently don't show up on G1 lines. Most numeric values aren't that big so there's no huge size savings on that, either.

 

The one thing binary is good at is avoiding having the firmware do the conversions from text to float. Not convinced that that's a major problem (makerbot is the only company to bother with binary - reprap's work fine without it) but it's probably more than noise..

 

> Printing from SD card lets the micronctroller not
> waste time reading and buffering from the serial port, and read vlovk
> by block from memory on the SD card

 

Most firmware I've seen buffer at least the next 5-6 commands - this is very easy to code and doesn't require lots of cpu.. Unless the commands are very small and very fast moves, even having a buffer of 5-6 is probably overkill.. As for acceleration, does the firmware ever look ahead more than the next move? I don't see why it would want to.. (Dan?)

[Michael Cook]

I'm not sure that's entirely correct. Bytes is bytes, ASCII or not..

True, but binary can be more compact that ASCII, aside from other issues.

Everything Jetguy said makes sense to me. There is a document in the Makerbot GitHub repository that provides the specification for exactly how s3g and x3g files work if anyone is interested.

From a quick skim it looks like s3g might be slightly lower level (i.e. one G code may cause multiple s3g commands). The biggest gain I would expect is that since you're using a binary file, you know that you need to execute command 87 with numbers 7, 12, and 42 as parameters.

If you were using ASCII G code you would have to read in a line (some unknown number of bytes), parse out the command (figure out it's a G102), find the parameters, convert them from strings ("12.72") to numbers, then execute them. All that takes a lot of time and memory for a small processor like an Arduino. Binary is easier, faster, and more compact. From two ulta-quick spot checks, it looks like s3g is ~20-30% smaller than the ASCII gcode file.

As for USB, the Arduino Mega used in a Thing-o-Matic acts as an old fashioned USB to serial adapter, capable of 115 kiloBITs per second. According to a post on the Arduino forum, a normal speed to read a file off an SD card is 250 kiloBYTEs per second, which is ~16x faster. I don't know if the Replicator boards can transfer data faster over USB.

[ddurant]

> If you were using ASCII G code you would have to read in a line

> (some unknown number of bytes), parse out the command (figure

> out it's a G102), find the parameters, convert them from strings

> ("12.72") to numbers, then execute them. All that takes a lot of

> time and memory for a small processor like an Arduino.

 

Almost like I said, except for the "takes lots of time and memory." It doesn't really take noticeably more memory and it doesn't take that much extra time.

 

> capable of 115 kiloBITs per second

 

Not exactly but close.. 115200 baud is about 11k bytes/second but, anyway, lots of RAMPS (which runs an Arduino Mega) people are using 250,000 baud since it's less error prone than 115200 baud.. Text gcode tends to be about 30 bytes per line so that's well over 350 lines/second at 115200 and over 650 lines/second at 200k baud.

 

How fast do you think machines consume gcode?

 

> a normal speed to read a file off an SD card is 250 kiloBYTEs per second, which is ~16x faster

 

Er... So? What if it could read a gigabyte per second? Would that be even better?

 

I'm not arguing against binary or SD cards - I just don't think they're as much of an advantage as people seem to think. Once toys like TinyG get here, printers using that will probably need binary and/or SD but todays fastest machines can be perfectly happy with plain ol' ASCII gcode over USB..

[Bottleworks]

Hmm...  The LinuxCNC group refuses to use USB because of latency. If I use USB with a rep1, it can sometime give glitches/short pauses.  Surely there is something to MBI and a major open source CNC controller, trying to avoid live commands over USB.  If you want the best results, always print off an SD card.  It not about speed or throughput, it's about latency and real time control. 

[Dan Newman]

Here's my take on the issue. The big savings isn't so much binary s3g vs. gcode,
but rather the computations which RepG offloads from the uProcessor as part of
generating the s3g.

In generating the s3g, RepG performs some floating point calculations which
otherwise would be done on the uProcessor

1. Convert units of millimeters to steps. The uProcessor does this as multiplies:
one per axis. If it's smart it only does it for axes which have to move in order
to execute the instruction. Typically that will be three axes (x, y, extruder)
but worst case it's five on a Rep 1. For an AVR with built in integer multiply,
that takes 140 cycles per multiply. So typically, that's 420 cycles and worst
case it's 700 cycles.

2. Determine the distance involved in the move. Since different axes may have
different steps per mm, you really need to do this calculation in units of mm.
If you do it in units of steps you hit what I call the "non-square pixel problem".
(Comes up when doing dithering for things like facsimiles which have a vertical
axis in units of lines per inch and a horizontal axis in pixels per mm -- gotta
love compromise in standard bodies -- and the aspect ratio of the pixel size is
either 1.04 or 2.07, depending upon the resolution.)

Anyhow, for the distance calc you have one multiply per axis involved, n-1 additions
where n is the number of axes involved, and a sqare root. That's
500 + (n-1)*110 + n*140 cycles and for a typical case (n=3) and worst case (n=5), that's
1140 and 1640 cycles, respectively.

3. Finally, there's decomposing the feed rate (magnitude of the velocity vector)
into the speed components along each axis and then applying per axis feed rate
limits. I don't recall how that breaks down, but it's easily 150 cycles for
typical case.

So, typical case the total cycles is 1710 cycles and worst case 2490 cycles.
On a 16 MHz processor, that's 0.11 ms for typical and 0.16 ms for worst case.

That may not sound like a lot, but there's a lot else going on on a Rep 1.
For instance at ~95 steps/mm for X & Y, to move at 120 mm/s, that's 11400
steps/s and if the stepper interrupt takes on average 60us, that's 0.684 s.
That leaves only 0.316 seconds for everything else: monitoring three temp
sensors, updating the LCD display, managing USB comms, and acceleration
planning.

With Sailfish, we've found that saving that pittance of 0.11 - 0.16 ms
actually helps when doing prints with lots of very tiny line segments (fine
detail): it helps reduce the incidents of the acceleration planner falling
behind. And, we actually introduced a new accelerated move s3g command
which offloads a bit more work onto RepG. Actually, what it did was provide
a s3g move command more apropos to acceleration. The old move commands
were geared towards non-accelerated moves and just said: execute these
steps at this step rate. You'd have to work backwards and figure out
what the target feed rate was in mm/s so that you could then impose
max accels in units of mm/s^2. (No point in doing things in units of
steps and steps/s^2 owing to the non-square pixel problem.)

Now, s3g isn't needed to do the above: it's possible to send the gcode
commands using units of steps instead of mm and to add a new G or M
command which provides the details useful to an accelerated planner.
So my point isn't that s3g is necessarily better than gcode. Rather
my point is that offloading calc onto your desktop (which has built
in floating point and a much faster processor) is beneficial. And
that's what I think RepG is bringing to the table and the use of
s3g can be argued to be a side effect. (You can still argue the utility
of that side effect, but I think it's a secondary benefit.)

Dan

[Dan Newman]

P.S. Folks who have looked at the Sailfish firmware know that inside it's
doing fixed point math in the accel planner and not floating point. So,
some/most of the below calcs are actually faster -- fewer cycles -- for
Sailfish in reality. That said, on a Rep 1 with the extruder control
no longer offloaded to a separate processor and the need to deal with five
1/16th stepping axes, we found that saving 1000+ cycles per gcode move
command was still very benficial.

Dan

> --
> You received this message because you are subscribed to the Google Groups "MakerBot Operators" group.
> To post to this group, send email to make...@googlegroups.com.
> To unsubscribe from this group, send email to makerbot+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/makerbot?hl=en.
>

[ddurant]

If s3g does all those calculations ahead of time on the pc, that's a definite savings in processing time for the Arduino..

 

...but I still say that USB has plenty of bandwidth and, if done right, low enough latency to drive all of todays printers. And a Mega has enough horsepower to do it.

 

> If I use USB with a rep1, it can sometime give glitches/short pauses

 

Instead of USB being slow, this is more likely something on your PC or environment causing problems. Maybe anti-virus software or some electroinic device causing noise. THIS is really the reason to print from SD, IMO, not speed/latency of USB or Arduino speed.

[Ethan Dicks]

On Mon, Nov 19, 2012 at 10:52 AM, ddurant <ddur...@gmail.com> wrote:
> If s3g does all those calculations ahead of time on the pc, that's a
> definite savings in processing time for the Arduino..

This is huge. Soft floating point on a 16 Mhz 8-bit processor is
going to be *slooow*.

> ...but I still say that USB has plenty of bandwidth and, if done right, low
> enough latency to drive all of todays printers. And a Mega has enough
> horsepower to do it.

It's not the wire speed that's the problem here, it's latency and
keeping the silo full.

Back in the "old days", with real serial ports, if your application
wanted to squirt bytes out of a serial port, an OS driver (or in the
DOS days, your application code since the BIOS drivers were so
terrible), pulled bytes from your buffer and slapped them into a real
I/O register and they went out at wire speed, one at a time. The real
problem there was interrupt latency from the OS. Later chips (16550A)
had internal buffers that could accept more than one byte from
(usually) the modem before triggering an interrupt (usually 12 bytes
received on a 16-byte buffer) so that you wouldn't lose bytes, but you
wouldn't be spending all your "time" rushing back to the modem byte
after byte with all the overhead.

With USB, there's a huge (by comparison to a simple serial driver)
networking stack that takes your buffer and converts it to a series of
network packets that are deconstructed by the chip on the MakerBot
motherboard (the FTDI chip or the ATmega u8) and turned into a simple
(1 start bit, 8 data bits, 1 stop bit) serial stream.

What matters is that the stream of bytes coming out of the FTDI chip
keep coming fast enough to prevent pauses in the motion of the print
head because they are waiting for their next set of coordinates. If
_that_ silo "starves", the head will pause until the next batch of
bytes assembles into a move command.

The clock speed on the USB wire is *not* the bottleneck. It's
substantially faster than the async serial speed between the FTDI chip
and the AVR processor on the MakerBot. The bottleneck, as far as I
understand, is PC host overhead and the amount of time and CPU cycles
it takes to take a wad of s3g bytes from your file and get them _onto_
the USB wire. This is why printing from an SD card is "pause free" -
the AVR chip can pull the bytes it needs when it needs them, and no
latency and overhead to do it.

In a sense, I'm restating what you are saying with "THIS is really the

reason to print from SD, IMO, not speed/latency of USB or Arduino

speed," but the part I'm adding is that the PC-side of USB _is_ slow,
or rather has high latency, too high to reliably keep the commands
coming to the AVR/Arduino uninterrupted.

-ethan