My job is paying me to work on this project despite the fact that it is
under the GPL, therefore the more requests you make, the longer I can
justify working on this instead of the (gulp) .Net system they've been
developing.
Below I've posted a little snippet of Ruport calling a query out of
the database itself (config and query not shown) and then generating a
PDF. This is actual functional code which prints out customer labels.
Also... scouts honor on a first release of Gambit (The game design
software James and I built) before the end of the summer.
With no further ado, a (rudimentary) Ruport example.
require 'pdf/writer'
pdf = PDF::Writer.new
pdf.select_font "Times-Roman"
pdf.margins_mm(12,0,20,0)
pdf.start_columns(3,27)
sql_stored( "RUPORT_QUERIES", "RECALL") do |row|
row["DUEDATE"] = row["DUEDATE"].to_s.slice( 0 .. 9)
row["ZIP"] = row["ZIP"].to_s.slice(0..4)
pdf.start_new_page if pdf.lines_remaining < 5
pdf.text "Due: #{row["DUEDATE"]}\n" +
"#{row["FIRSTNAME"]} #{row["LASTNAME"]}\n" +
"#{row["STREET1"]}\n" +
( row["STREET2"].eql?("") ? "" : row["STREET2"] + "\n" ) +
"#{row["CITY"]}, #{row["STATE"]} #{row["ZIP"]}\n\n" +
( row["STREET2"].eql?("") ? "\n" : "" ),
:font_size => 11, :justification => :full
end
<%= pdf.render %>
whoops... should be: pdf.render
Thanks James for pointing this out
<snip>
Funny you should mention it. I've created a generic reporting framework
for use at work.
I start with a predictable filesystem layout:
report/
archive
bin
lib
log
mail
sql
Then I use a predictable naming convention. If you have a script called
"foo.rb", then it automatically uses log/foo.log for logging, looks in
mail/foo.mail for any email addresses, and looks for all related sql in
sql/foo.sql. All configurable of course.
It's setup to make automatic handling of database connections, preparing
sql statements, email, logging, and archiving as painless as possible
(for me, anyway).
For the sql itself, I use a specific format. Each sql statement is in
its own paragraph and tagged with a comment. That comment is used by
method_missing behind the scenes to dynamically two seperate methods,
one that returns the sql statement itself (a string) and one that makes
the call out to the database.
So, for example, if you have a sql statement in the foo.sql file that
looks like this:
--main
select *
from foo
Then you get a Report#main_sql and Report#main_data method. The former
just returns the sql, while the latter makes a call out to the database.
Here's a typical example for a simple report:
# foo.rb
require "report/genericreport"
report = GenericReport.new
report.start_log
report.init("db","foo") # Connect to db, prepare all sql statements
# Use the logger instance directly
report.logger.info{ report.report_name + " started at " + Time.now.to_s
}
# Execute the main query
report.main_data{ |rec|
File.open(report.csv_file, "w+"){ |file|
file.puts(rec.join(", "))
}
}
# Sends an email to everyone in mail/foo.mail
report.send_email("report finished")
# Move the .csv file to the archive directory
File.move(report.csv_file, report.archive_dir)
report.logger.info{ report.report_name + " finished at " + Time.now.to_s
}
# Cleans up all database and statement handles and the logger instance
report.finish
# END foo.rb
There's more to it than that, but that's the basic idea. I've set it up
to do some handling of CSV files, since that's mostly what we use (and
Excel reads them fine). I've never been asked to put a report in PDF
format, actually. Any
reports that wanted special formatting like that I would probably handle
separately, or create a subclass that handled it.
Anyway, is something like that sorta what you had in mind? I never
released it because I kinda figured everyone liked to do reporting their
own way. No?
Regards,
Dan
> Funny you should mention it. I've created a generic reporting
> framework
[snip very interesting project description]
> I've set it up to do some handling of CSV files, since that's
> mostly what we use (and
> Excel reads them fine). I've never been asked to put a report in PDF
> format, actually. Any reports that wanted special formatting like
> that I would probably handle
> separately, or create a subclass that handled it.
I've done quite a bit of this kind of thing for one company I work
for. We CSV the data into Excel and then just hit it with formatting
macros to pretty it up. I've looked at using Excel's XML format in
the past, which allows most formatting, but its lack of chart support
(when I last checked, some time ago) was a show stopper for my company.
> Anyway, is something like that sorta what you had in mind? I never
> released it because I kinda figured everyone liked to do reporting
> their
> own way. No?
I can't speak for everyone, but I do a lot of custom reporting in my
work (which is what led me to feed Greg the hack in the first place)
and I find your software very interesting. I would download it and
play with it, if released.
James Edward Gray II
<snip project details>
>Any reports that wanted special formatting like that I would probably handle
>separately, or create a subclass that handled it.
<snip>
> Anyway, is something like that sorta what you had in mind? I never
> released it because I kinda figured everyone liked to do reporting their
> own way. No?
Yes. Everyone does like to do reporting their own way :)
Ruport's goal will be to facilitate this, not get in the way of it.
I'm basically going to collect all the little things we all need to do
that get annoying and put them together in a generic framework so that
you can have the low level drudgery done for you and focus on the
actual task. This is similarly inspired by Jame's HighLine library and
it's purpose. Ruport is not going to be a massively heavy
auto-generator, just a tool to help along the way.
If you publish your code, I know you'll have at least two people
playing with it already.
If you do so, I encourage you to be nice and make it GPL compliant so I
can steal all your good ideas.
As far as the PDF example goes... that was a pretty bad example. I
promise a good one once Ruport is more well formed ;)
This is probably pretty far OT for your current project, but I'll mention it
anyway, in the interests of saving you from .Net ;-)
I need something that would take text in TWiki markup (with some variations)
and convert it to (X)HTML. The source of the text could be (and ideally is)
anything (from "plain text" files (with markup) to similar (marked up) text
stored in a database or directly from a plain text editor (like nedit)).
Ideally, it would also feed into an (X)HTML renderer / browser that would be
"under the control" of the "something". In other words, sort of like a push
technology--the HTML renderer/browser displays what it is told to display
(what it is sent, when it is sent) rather than having the user work in the
browser to, for example, change pages.
Initially (but hopefully not for long), the HTML renderer might be limited to
displaying text sent to it. Later it would be nice:
* if it was closer to a real browser such that it had the tabs feature, and
could follow links on the "remotely controlled" page in new tabs
* it could display pictures and so forth (beyond plain text)
I've noted the recent thread where Hal Fulton is looking for a simple HTML
renderer and will be following that with interest.
regards,
Randy Kramer
<snip>
> I've done quite a bit of this kind of thing for one company I work
> for. We CSV the data into Excel and then just hit it with formatting
> macros to pretty it up. I've looked at using Excel's XML format in the
> past, which allows most formatting, but its lack of chart support (when
> I last checked, some time ago) was a show stopper for my company.
The Spreadsheet::WriteExcel port I did was originally for reporting
needs. However, I decided that PHB's wasted too much of my time with
inane crap like, "Can you make the column headers blue?". From now on
they get CSV files only. Nyah.
> I can't speak for everyone, but I do a lot of custom reporting in my
> work (which is what led me to feed Greg the hack in the first place)
> and I find your software very interesting. I would download it and
> play with it, if released.
I'll try to get something out tonight or tomorrow. Otherwise it will
have to wait a while due to work issues.
Regards,
Dan
PS - The html-table package was also written with reports in mind. :)
<snip>
> I can't speak for everyone, but I do a lot of custom reporting in my
> work (which is what led me to feed Greg the hack in the first place)
> and I find your software very interesting. I would download it and
> play with it, if released.
Ok, I've put something together.
Download it from
http://rubyforge.org/frs/download.php/5524/gruf-0.0.1.tar.gz
Regards,
Dan
> Ok, I've put something together.
> Download it from
> http://rubyforge.org/frs/downl oad.php/5524/gruf-0.0.1.tar.gz
Neat! I will look at this more later but it definitely has some of the
things I'm looking for. Is it okay if I integrate some of this into
Ruport?
Go ahead. :)
Dan
>Go ahead. :)
Thank you Daniel. You will be given credit for anything I use. Anyone
who has any ideas or suggestions for a reporting engine in pure ruby
feel free to speak up as my first version will be arbitrarily released
on August 28th on rubyforge. (Project name: Ruport)
Thanks for sharing this. I'm playing with it and getting a lot of
good ideas. :D
James Edward Gray II
Thanks. I was just thinking that I should add a "config" directory for
those things you can't really hard code in a generic way, such as a
mailhost, etc. I'd use YAML.
Regards,
Dan
This exists in Ruport :)
The more I look through your code, the happier I am that you have
released it.
Thank you!
When working on PDF::Writer, I deliberately chose the MIT-style licence
because I wanted it to be used as widely as possible. It can be used in
GNU GPLed projects without problems (modulo issues with Adobe's patents,
but that's a generic issue, not a GNU GPL issue) or under commercial
projects without problems.
If one makes the report generator GPLed, then you have severely
restricted it. If one makes the report generator MPLed (for example),
then the commercial and other OSS opportunities are greater, but the GPL
is incompatible with the MPL (this is mostly the GPL's fault).
Ultimately, the licence is up to you, but life is easier if you choose a
single minimally restrictive licence rather than dual-licence. (Don't
get me *started* on the licensing for Text::Hyphen because of jackasses
who licenced their hyphenation files under the GNU GPL.)
-austin
--
Austin Ziegler * halos...@gmail.com
* Alternate: aus...@halostatue.ca
+1. I BSD license my projects whenever possible - if someone wants to
use them in a product, more power to them. More than likely, they'll be
feeding bug fixes and whatnot back into the code, and even if not, it's
more mindshare for the project. It's worked out well for me in the Java
world with PMD - http://pmd.sf.net/ - good times.
Yours,
Tom
> Ultimately, the licence is up to you, but life is easier if you choose a
> single minimally restrictive licence rather than dual-licence. (Don't
> get me *started* on the licensing for Text::Hyphen because of jackasses
> who licenced their hyphenation files under the GNU GPL.
Okay, I'm fairly certain that I will not be releasing under just the
GPL, as it's clear that people think that it's not a great idea. Thank
you for your suggestions Austin, they were helpful.
I am going to consider releasing under the Clarified Artistic License.
Given the choice between this and the disjunctive license of Ruby, what
are your opinions. Anyone feel free to respond on or off list.
Neither the Ruby licence nor the Clarified Artistic licences are
"approved" by the OSI (www.opensource.org). That said, the Ruby
licence is pretty clear as to what's allowed (pretty much "anything").
I still think that your best choice is going to be BSDish or MITish,
but if you go with anything else, then you'll probably need to do
CAL/GPL disjunctive or the Ruby licensing option.
Given that it's a Ruby package, the Ruby licence is probably okay enough.
> Neither the Ruby licence nor the Clarified Artistic licences are
> "approved" by the OSI (www.opensource.org).
That's not true. The 8th clause is the one that makes the Artistic
license GPL compatible, and the approval by OSI is listed below:
http://www.opensource.org/licenses/artistic-license.php
> That said, the Ruby
> licence is pretty clear as to what's allowed (pretty much "anything").
> I still think that your best choice is going to be BSDish or MITish,
> but if you go with anything else, then you'll probably need to do
> CAL/GPL disjunctive or the Ruby licensing option.
I am honestly not interested in using a BSD or MIT style license. I
understand there is a strong tradition in the Ruby community to lean
this way, and I respect this, but I am in favor of the FSF's ideals and
am looking to preserve them as much as possible while still offering a
fair and pragmatic licensing scheme for my software.
If that's the clause that makes the difference between the CAL and the
AL, then you're right. However, I was looking at the title.
>> That said, the Ruby licence is pretty clear as to what's allowed
>> (pretty much "anything"). I still think that your best choice is
>> going to be BSDish or MITish, but if you go with anything else, then
>> you'll probably need to do CAL/GPL disjunctive or the Ruby licensing
>> option.
> I am honestly not interested in using a BSD or MIT style license. I
> understand there is a strong tradition in the Ruby community to lean
> this way, and I respect this, but I am in favor of the FSF's ideals
> and am looking to preserve them as much as possible while still
> offering a fair and pragmatic licensing scheme for my software.
Ah. That's where we differ. I disapprove of RMS and the FSF's methods
(not so much the ideals) so strongly that, given a choice, I will not
licence anything GPLed and if there's a choice between a GPLed library
and a non-GPLed library, I will preferentially use the non-GPLed library
even at the possible expense of features. I have no issue with the GNU
GPL *as a licence philosophy*, but I have major problems with the
politicised preamble that I do not wish to redistribute. Ever. I also
have major issues with the denial of the GNU GPL as a viral licence
(that's a feature of the licence, not a negative point), the whole
"software freedom" issue (as if software were something that could be
emancipated), and in general the deception that goes on surrounding the
GNU GPL.
IMO, there's similar licences (the Mozilla PL, for one) that are
friendlier, stronger, and have no political crap surrounding them. IIRC,
there's a licence that is a variant of the GNU GPL simply with the
preamble removed.
Note that disjunctive licensing is questionable at best, at leaves your
users open to choose whichever licence they want. Strictly speaking,
although I've preserved the intent of several packages (e.g.,
Text::Format, others) by keeping the GPL as one of the specified
licences, but I could have *simply* chosen to change the licence for the
Ruby version as the Artistic licence, completely ignoring the GNU GPL's
existence on the Perl version of Text::Format.
Not trying to be difficult here or even start a flamewar, but trying to
explain why I prefer the other licences to the GNU GPL.
I won't use this post as a soapbox for my support of the FSF so I won't
attempt to defend the GPL on list. However, it's safe to say that I do
believe in their philosophy, and I find the preamble to be one of the
best parts.
> Not trying to be difficult here or even start a flamewar, but trying to
> explain why I prefer the other licences to the GNU GPL.
I appreciate your honest opinion. I am fairly certain that I will use
the license of ruby. I.e. a disjunction between the GPL and the Ruby
license. This way no one needs to deal with the GPL if they don't want
to, and anyone who DOES want to support the use of the GPL may do so
freely. I feel that this is a fair and reasonable course of action for
someone who is most certainly a Free Software (and not necessarily an
Open Source) advocate.
Will anyone have trouble using Ruport under such a scheme?
Without looking at Ruport, I added support for YAML config files in
gruf. So, say you have a config file that looks like this:
# foo.config
host: blah.com
mailhost: bar.com
ENV:
LD_LIBRARY_PATH: /usr/lib
ORACLE_HOME: /opt/oracle
Then you call Report#configure. This will automatically generate
"host" and "mailhost" methods (accessors) for you, with the values set
appropriately. The ENV tag is special in that it simply sets
environment variables for your script (which I need sometimes).
Checked into CVS. :)
Regards,
Dan
I'll happily use it. (Just to be clear.)
I can't speak for Tom, but I've actually never earned a penny from my
open source projects. Not that I wouldn't *love* to, but for me, it
isn't about making money from these projects. Let me briefly go through
the various projects that I have and *why* I did them:
1. MIME::Types, Text::Format: These two were originally done for a tool
called RTidy/CD which ran HTMLTidy for the Fog Creek web site
editor/manager CityDesk (hence the /CD). I needed to format the
output of the tool nicely and handle binary files differently than
HTML files, so I ported these from Perl. RTidy/CD itself is open
source, but no one ever asked me for the source and CityDesk version
2 made RTidy/CD obsolete. I keep maintaining MIME::Types and
Text::Format because I have taken them on as responsibilities.
Text::Format saw a 1.0 release recently and probably won't see a lot
of work until Ruby 2.0 (or at least the M17N strings in Ruby 1.9)
comes out and I need to look at what may need to change for that.
MIME::Types is likely to see more releases, plus a change in the way
that the whole thing works so that the core library (MIME::Type, and
the MIME::Types lookup) can be released separately from the MIME
content type data. I will also be looking at integrating
libsharedmime and libmagic functionality. I should get a new version
out soon because I have noticed that the definitions are a little
out of date, but that depends on the other work that I do.
2. Uninheritable. This library will probably never see another release,
and I'd be surprised if anyone uses it. It was more of a proof of
concept and there seemed to be a rash of requests for making classes
"unable to be inherited from" at the time that I did it originally.
3. TeX::Hyphen, Text::Hyphen: These libraries were created because of
needs for Text::Format. Martin DeMello actually ported TeX::Hyphen,
and then I modified that to Text::Hyphen (with a lot more languages
supported) to fix some bugs that existed in the original Perl (e.g.,
not Martin's fault, and VERY complex to fix, hence the new library as
I wasn't interested in creating a TeX interpreter in Ruby). The
former probably won't see any more updates by me, although Martin is
an administrator/developer on that project so he may choose to do
something with it. Text::Hyphen may see periodic updates as people
report hyphenation issues or point out that there's an updated
version of a language that I don't have converted. It will also
probably see an M17N update when Ruby has M17N strings. After that,
the library Just Works, so I expect that it will probably see few
releases.
4. Transaction::Simple: Created for the initial (technology preview)
release of PDF::Writer. It's grown a bit since then, and there are
things that I *want* to do with it that I just haven't had time to
ask how they might be done of people who are much better at Ruby than
I am (one of the big flaws of Transaction::Simple is that it can end
up duplicating objects, and this has caused an API change in
PDF::Writer that I ultimately want to change back). This will see
more releases.
5. color-tools: Created for the second release of PDF::Writer (1.0) to
collect colour information in one place. It's grown since then, and
will probably continue to grow. I'm looking at incorporating
littleCMS-alike functionality (and possibly bindings to littleCMS for
performance) so that it can deal with ICC colour profiles and then
properly convert between various colour spaces meaningfully and
reversably (not all colour conversions are reversable right now).
This will see more releases.
6. Diff::LCS: Originally written to support Ruwiki. Interestingly, this
library shouldn't even have to exist, since Algorithm::Diff did most
of what I needed at the time (Diff::LCS does more, now) but was
licenced as GPL, which was not compatible with the desired Ruwiki
licence. It will see more releases, probably ultimately including
better diff support and merge support.
7. Archive::Tar::Minitar: Adapted from code provided by Mauricio
Fernandez for Ruwiki's command-line deployment support. This will
also see more releases, as I plan on adding symlink and hardlink
support, but will probably stop at that point.
8. Ruwiki: Picked up because I needed a wiki. I have plans for further
work on Ruwiki, but the changes I plan are big, and I will be looking
at the implementations of some other wikis in the meantime. This will
probably be dependent upon changes to Diff::LCS moving forward. I'll
probably be looking at the AJAX stuff that one of the Rails wikis
just added, but Ruwiki will not be moving to a web framework (it will
still be a CGI-based Wiki) and will remain flat-file based at its
core. Someone -- I *think* Gavin Kistner -- did something with node
reparenting that I may be digging into, or even reusing the OWL stuff
for future versions. There have been changes in the Ruby environment
since I last worked on Ruwiki -- Ruwiki will adapt, and I will make
it possible to upgrade.
9. PDF::Writer: This was originally written because there was a hole in
Ruby's support for PDF. It's somewhat taken over my life at this
point, and as I pointed out in a different post today, I'm working on
a release right now (well, I'm procrastinating right now) and then
I'm starting work on version 2.0 -- which will be demonstrated, in
part, at RubyConf 2005 because I'm also adding a slideshow producer.
Don't get me wrong -- if anyone wants to donate to the "Encourage
Austin" fund, I'll not turn it down. It'll help defray my costs to
RubyConf ;) I do this because I *love* it, and it keeps me thinking
about different programming mechanisms and paradigms that I don't get to
work with at my day job. This is *very* valuable, because I've been able
to present ideas that have been accepted even though they're a bit
"off-the-wall". Because I love doing this, why should I be selfish? I
share what I do.
> Without looking at Ruport, I added support for YAML config files in
> gruf. So, say you have a config file that looks like this:
You'd need to break into some computers to see Ruport just yet :) It
hasn't been released yet! However, the development head will be moved
to CVS on rubyforge when the first release comes out. So it's only
going to be a secret until the 28th
>
> # foo.config
> host: blah.com
> mailhost: bar.com
> ENV:
> LD_LIBRARY_PATH: /usr/lib
> ORACLE_HOME: /opt/oracle
>
> Then you call Report#configure. This will automatically generate
> "host" and "mailhost" methods (accessors) for you, with the values set
> appropriately. The ENV tag is special in that it simply sets
> environment variables for your script (which I need sometimes).
Very neat! I'll definitely look into something like this for Ruport.
I don't handle ENV variables yet, though my config setup (or Jame's
actually) is essentially the same.
-Greg
Austin put this quite well though I have to say that I've made my
living off of free software thus far. Essentially I have been funded
by companies to implement certain features into generic systems that
they don't have much of a stake in. I make this agreement with them
and all they care about is the fact that they get a product that does
what they want, and they leave me free to share.
To me, anything I can share with the community, I will. I do not take
straight proprietary jobs simply because I never want to lock my source
in a vault for life and throw away the key. There are a ton of reasons
to write Free Software, and Austin mentions many of them.
For me, it's just fun to share and get to know people and all of that
good stuff.
I've put release 0.0.2 out there which has the config support, and a
few other minor tweaks and fixes.
http://rubyforge.org/frs/download.php/5594/gruf-0.0.2.tar.gz
Regards,
Dan
Here are my .02 eurocents about a generic reporting tool:
i would probably split it into 2 modules: data feeders and report generator/formatter. This way you
will be able to find feedback from more sources for both modules. Moreover you will have a very
clean separation of concerns.
:alex |.::the_mindstorm::.|
> Here are my .02 eurocents about a generic reporting tool:
> i would probably split it into 2 modules: data feeders and report
> generator/formatter. This way you
> will be able to find feedback from more sources for both modules. Moreover you
> will have a very clean separation of concerns.
>
> :alex |.::the_mindstorm::.|
This is a great idea and I will do my best to make Ruport model this
type of separation. Ruport will be on CVS sometime later this week for
a little scrutiny before it's initial release on the 28th, I will post
when it is up.
It's over at rubyforge:
http://rubyforge.org/survey/survey.php?group_id=856&survey_id=20
It'll determine a few key aspects of how I spend the next two weeks or
so, whether or not Ruport gets up on CVS before the 28th, whether or
not documentation will be included with the first release, whether or
not it will be split into two modules, and whether or not I'll consider
a license that's even less restrictive or complicated than the Ruby/GPL
combo. So if any of this matters to you, feel free to come and let
your opinion be known in statistical form.
It also lets you submit feature requests :)
Glad you like the idea.
Also passing through the survey I have noticed the 2nd question that is (again) imho quite tricky
:-). Definitely it should be easy to use to gain a community; once you have it you will need to
raise the bar and make it more and more customizable. This is something that I would say that fits
best: with start small philosophy ;-).
Even if I am a Ruby freshman I would be really happy to be able to see the sourcebase very soon and
even more to help.
cheers,
:alex |.::the_mindstorm::.|
I've updated this survey and it has nearly double the questions and all
the yes or no's have been changed to sliding scale, so you can be a
little more wishy washy with your feelings.
Linkage here:
http://rubyforge.org/survey/survey.php?group_id=856&survey_id=21
A couple important things added to the survey ask about your OS and
database just so I can see at of curiosity what systems and databases I
should prioritize my support for (though with all luck Ruport will be
cross platform). I usually stick it to Windows users when in doubt,
but for Ruport this will not happen because the company that's helping
fund it is reliant on a Windows farm, so for once I'll cater to you M$
guys out there ;)
I will try to stop soliciting Ruport now, though all of the discussion
it has generated has become addictive. Hope you like what I have to
offer come the 28th.
If anyone is interested in taking a walk through the manual to both
test out Ruport and help improve it before it's first release, please
feel free to let me know on or off list.
I will not post again tomorrow announcing the release of the manual,
because I've spammed ruby-talk enough already. But you will be able to
find it at:
There will be some news articles regarding development as well on this
weblog:
Any help would be greatly appreciated. I'd love to see suggestions on
how to improve the setup and install process but just testing them out
to see if they work would be quite helpful. For people fearful that
Ruport will be impossible to install, do not run away yet,
the features that require excessive effort are basically add on
features, and you will still be able to use Ruport for most of your
general reporting needs. The special features are cool though, and I'd
like to make them easy to enable.
I'll announce Ruport on Sunday when it's complete. Otherwise, please
let me know if you're interested in being a test driver and I'll do my
best to get the manual up tomorrow afternoon.
-Greg