convert vector to raster without getting anti-alias lines

398 views
Skip to first unread message

Tyrzi

unread,
Feb 14, 2018, 10:21:32 AM2/14/18
to OpenToonz Users Forum
When converting vector lines to raster, opentoonz by default makes raster anti-aliased. But I don't waaaana. If there isn't an option for making it alias then I think it would be a good addition. 

Also since we are at it, it would be nice if there is a render option called "vector alias render" that renders all vector images without anti-alias.

Rodney

unread,
Feb 18, 2018, 8:30:01 PM2/18/18
to OpenToonz Users Forum
Can you provide a few images of what you are seeing?
I started to investigate this but kept getting sidetracked as I didn't have anything specific to look into.
(My sidetrack took me more into premultiplication territory... which could relate to what you are seeing)

Tyrzi

unread,
Feb 18, 2018, 8:59:19 PM2/18/18
to OpenToonz Users Forum

Zachary Brown

unread,
Feb 19, 2018, 1:00:14 AM2/19/18
to OpenToonz Users Forum
I don't have a suggestion here -- just wanted to say that was a very cute animation. I liked the story!

Be well,
Zack

Rodney

unread,
Feb 19, 2018, 1:51:42 AM2/19/18
to OpenToonz Users Forum
So, my guess is...

Change the Camera Settings DPI setting to a higher number.
It appears that when Vectors are converted to Toonz raster this setting is factored into the equation.
Also, just in general, make sure that setting isn't set too low AND not too high either.
If too low... you'll see the pixelization.
If too high... OpenToonz will take a lot longer to render to preview and continuously update frameds.
Not to mention a higher DPI setting will take forever to final render.


As an aside, I believe technically that DPI is actually PPI (pixels per inch).
DPI (Dots per inch) is a hold over from the dot matrix printer days and has more to do with printed imagery on paper.
It's a little more complicated than that but... yeah... 
I think DPI is maintained in OpenToonz primarily because it would confuse too many people to change it.

Here are two frames where the only thing different was the DPI setting in the Camera Settings when the image was converted from vector to Toonz raster:
I should have exaggerated the setting even more so that it's easier to see.  Some of that artifacting/antialiasing was probably smoothed also when I rendered those two frames out at 200DPI.
200DPI is probably overkill for most of my needs.





Tyrzi

unread,
Feb 19, 2018, 2:01:34 AM2/19/18
to OpenToonz Users Forum

I dunno how to change DPI to the highest setting, but lemme ask, does it make the picture to have absolutely no anti-alias smoothing? Because your example still seems to have anti-aliasing on the sharper picture. The end result I'm looking for is the same you can get when you watch an SWF file and put the quality to low, you can see pixels crisp and clear, like in this piece I made. 


The render could have been done in flash with ease by disabling smooth in the PNG export etc.

Rodney

unread,
Feb 19, 2018, 2:30:11 AM2/19/18
to opento...@googlegroups.com
All I know is that the effect of changing that setting is pretty dramatic.

If you can supply a test image I'll run it through and see what comes out.
(I'll test with your run cycle image)

DPI.jpg

Tyrzi

unread,
Feb 19, 2018, 2:35:16 AM2/19/18
to OpenToonz Users Forum
It's not good enough if there is even a little bit of that anti-alias smoothness. Oh well, I may be able to figure something with this knowledge in my arsenal. So how to change DPI?

Zachary Brown

unread,
Feb 19, 2018, 2:45:36 AM2/19/18
to OpenToonz Users Forum
Hi Tyrzi,

I remember this one from my question about the plastic tool.

In the XSheet drop-down menu, choose "Camera Settings". You can set the DPI there.

Be well,
Zack

Rodney

unread,
Feb 19, 2018, 2:53:10 AM2/19/18
to opento...@googlegroups.com
In the XSheet drop-down menu, choose "Camera Settings". You can set the DPI there.


Or at the very top of the screen where the main menu says 'Xsheet' the first two options are Scene Settings and Camera Settings.
The DPI setting can be accessed there (via Camera Settings) as well.


As for not having any antialias/smoothness... that is a product of any raster image.
Your gif animation for instance has a ton of those because the gif format is pixel based.
The only way to stay away from that is to stay with vector formats (SWF is one, SVG is another).
OpenToonz internal format is .PLI.


Added:   I should mention that there is a known problem with premultiplied alpha channels in OpenToonz so that can be an issue when you are using transparency.


I see where we can create drawings in Toonz Raster without antialiased lines (via "No Antialiasing') but that doesn't help us with conversion.


Tyrzi

unread,
Feb 19, 2018, 3:04:03 AM2/19/18
to opento...@googlegroups.com
yea I understand the difference between pixel and vector. I was just wondering that if it's possible to add an option that converts vector to raster without smoothness. I'm not really obsessed with this option but it would make things easier for me since vector always has anti-alias

Rodney

unread,
Feb 19, 2018, 3:08:16 AM2/19/18
to OpenToonz Users Forum
 I was just wondering that if it's possible to add an option that converts vector to raster without smoothness.

Do you mean 'with' smoothness.
Isn't the issue with the jaggies one of not being smooth enough?

The classic approach to resolving that is to uprez the render/conversion and then scale the result down to effect the smoother line.

Can you share an image that compares a vector to raster conversion that has the result you want?

 

Tyrzi

unread,
Feb 19, 2018, 3:16:29 AM2/19/18
to opento...@googlegroups.com

No, I meant "without", because by default vector converts to raster with smooth lines, and the idea is that I want it to be crispy pixelated and I don't want any smoothness 


Rodney

unread,
Feb 19, 2018, 3:25:46 AM2/19/18
to opento...@googlegroups.com
I follow you now..
Thanks for the example!

There are a few places in OpenToonz where antialiasing can be turned on/off or adjusted.
I have not seen that lead to any reasonable success.

Hopefully someone else has some insight into this.

There is a "Use DPI for higher calculations" setting in Preferences on the Drawring tab.
I don't think that will do the trick but I'll investigate.



Noah Farron

unread,
Feb 19, 2018, 6:49:03 AM2/19/18
to OpenToonz Users Forum
How about...

Convert your vector lines to Raster instead of Toonz Raster, then run a cleanup job on the raster level.

Open the Cleanup window, and select the Line Processing type that meets your needs, switch Antialias to None.

Go to View > Preview Cleanup (I assume that's where it is, I use the Ghibli rooms, sorry) to preview what the result will look like directly in the viewer. Turn it off again when you're done previewing then run the cleanup, I think in Default rooms layout that's under Scan > Cleanup.

Tyrzi

unread,
Feb 19, 2018, 7:19:48 PM2/19/18
to opento...@googlegroups.com

Thanks for making me look into cleanup section because I've never been there. I could use it for something but not for this.

As you can see, the intersecting parts don't really work in my favor since sure I have aliased lines, but the cleanup still tries to find color differences and between red and green there is some other shades and clean up makes em black (or other color I choose). Also cleanup doesn't separate color to Line and Area and I need those modes to make my work easier. Sure I could check every frame separately and fix those black spots until it's what I want, but I could do the same task easier and faster with other methods that too are limiting my Line/Area modes.

Jane Eyre

unread,
Feb 19, 2018, 11:07:25 PM2/19/18
to OpenToonz Users Forum
If your colors are fully opaque, you can actually remove antialiasing pretty successfully in the FX - without needing to convert the level at all. For your vector level, try adding both “sharpen” and “Density Ino” in the FX schematic. Both are under “Image Adjust”. I’ve gotten that combination to completely remove antialiasing around the outside edge of a vector drawing. In areas of overlapping, the pixels aren’t always exactly perfect, but it’s very very close.

Density should be all the way up to 10 and does not need a reference level. For Sharpen’s intensity, the best value depends on your image but sometimes can be fairly low or as high as 3000.

Tyrzi

unread,
Feb 19, 2018, 11:33:40 PM2/19/18
to opento...@googlegroups.com

Jane, can you demonstrate with pictures that what you mean? Because I have some issues with how I understood you.


Noah Farron

unread,
Feb 20, 2018, 6:37:33 AM2/20/18
to OpenToonz Users Forum
The cleanup method works for me, you need to adjust the White Thres[hold] on the black color (to fix the black overlap) and increase the Hue Thres[hold] on the other colors. The line "on-top" depends which line has the highest hue value in the overlap. The ideal method would be to use no transparency at all.


Jane Eyre

unread,
Feb 20, 2018, 2:20:19 PM2/20/18
to OpenToonz Users Forum
Using fx is definitely not perfect, especially with translucent colors, but can get close - depending on the image and your settings.
7C17C7B8-92AA-4F6B-9906-65A8F6FB4F31.png
462FC383-9AED-4657-A0E5-9B69BC7FB275.png

Tyrzi

unread,
Feb 21, 2018, 12:54:48 AM2/21/18
to OpenToonz Users Forum
getting close is not good enough. It should be so accurate that using fill tool in MSPaint can recolor the whole area without leaving little antialias artifacts.
Reply all
Reply to author
Forward
0 new messages