Hello,
Thank you for your interest in our work.
GlycReSoft can build a glycan database in a few ways. I'll assume you're using the Graphical User Interface, though everything I'll show you is doable from the command line.
Under the "Build A Glycan Search Space" menu, you get a set of tabs which support different strategies: "Pre-generated", "Combinatorial", and "Text File"
The combinatorial approach is the oldest, and includes few assumptions. You specify the minimum and maximum number of a given monosaccharide you are interested in in the table on the left, and specify any constraints you want enforced in the table on the right. You can use most IUPAC names for monosaccharides in these slots, or specify loose substituents by prefixing their names with a @-sign (@sulfate, @phosphate, @amino, etc.). The default bounds are set up for human N-glycans. While this method can generate all reasonable glycan compositions, it includes many improbable compositions too. It can work well when you don't know what you're looking for.

The pre-generated databases are databases created by me using biosynthetic simulation to explore the range of biologically feasible glycan structures, collapsed to compositions. You can check multiple boxes here to include both N- and O-linked glycans in the same search space, though keep in mind that searching for both can make things pretty slow.
The text file option lets you create a file containing exactly the glycan compositions you want. The syntax is a bit different from Byonic's (though that's a good idea to add support for).
Each line in the text file you create must have a glycan composition, followed by a tab followed by the glycan type (N-glycan, O-glycan, GAG-linker). An example would be:
{Hex:5; HexNAc:4; Neu5Ac:1} N-Glycan
{Hex:5; HexNAc:4; Neu5Ac:2} N-Glycan
{Fuc:1; Hex:5; HexNAc:4; Neu5Ac:2} N-Glycan
{Fuc:2; Hex:6; HexNAc:5; Neu5Ac:1} N-Glycan
{Fuc:1; Hex:6; HexNAc:5; Neu5Ac:2} N-Glycan