Having the need to synchronize some data with external sources, I have
written some tasks on Grunt to do so. I wonder if Grunt is actually the
tool to write batch jobs in, or there actually is some other alternative.
I have come across Node.io, but I rather stick to Grunt in this case. Do
you guys know about some alternatives, please?
Can you share some more details on what you are trying to accomplish?
You have talked about tools (both of which I am unaware of) and said
"batch", but apart from that there wasn't much more that could help anyone
make an informed suggestion.
Let me know, I'll try to help :)
Nuno
On Fri, Oct 5, 2012 at 11:08 AM, Guilherme Pim <pimguilhe...@gmail.com>wrote:
> Having the need to synchronize some data with external sources, I have
> written some tasks on Grunt to do so. I wonder if Grunt is actually the
> tool to write batch jobs in, or there actually is some other alternative.
> I have come across Node.io, but I rather stick to Grunt in this case. Do
> you guys know about some alternatives, please?
I am crawling some web pages to populate a database, and currently doing
that with Grunt (http://gruntjs.com/), which is a task-based 'build tool'
(that gets me confused, is it supposed to solely build?).
So, I have created some scripts in the following manner:
========== Task file ==========
module.exports = function (grunt) {
grunt.registerTask('courses/update-info', 'Updates all courses info on
the database', ... )
}
============================
So then I may have a somewhat controlled access to these jobs from the
terminal, with ability to list all existing jobs and their descriptions.
Also, I may pass some parameters to each task, although it's not very
readable because they can't be associated with a name:
grunt courses/update-info:4:5:2 (these will be passed as the tasks's
function arguments)
I guess I could go about creating node scripts and using a parameters
library along with console colors and stuff, but I would rather find a
framework, if such exists.
> Can you share some more details on what you are trying to accomplish?
> You have talked about tools (both of which I am unaware of) and said
> "batch", but apart from that there wasn't much more that could help anyone
> make an informed suggestion.
> Let me know, I'll try to help :)
> Nuno
> On Fri, Oct 5, 2012 at 11:08 AM, Guilherme Pim <pimguilhe...@gmail.com>wrote:
>> Hi guys,
>> Having the need to synchronize some data with external sources, I have
>> written some tasks on Grunt to do so. I wonder if Grunt is actually the
>> tool to write batch jobs in, or there actually is some other alternative.
>> I have come across Node.io, but I rather stick to Grunt in this case. Do
>> you guys know about some alternatives, please?
> I am crawling some web pages to populate a database, and currently doing > that with Grunt (http://gruntjs.com/), which is a task-based 'build tool' > (that gets me confused, is it supposed to solely build?).
> So, I have created some scripts in the following manner:
> ========== Task file ==========
> module.exports = function (grunt) {
> grunt.registerTask('courses/update-info', 'Updates all courses info on > the database', ... )
> }
> ============================
> So then I may have a somewhat controlled access to these jobs from the > terminal, with ability to list all existing jobs and their descriptions. > Also, I may pass some parameters to each task, although it's not very > readable because they can't be associated with a name:
> grunt courses/update-info:4:5:2 (these will be passed as the tasks's > function arguments)
> I guess I could go about creating node scripts and using a parameters > library along with console colors and stuff, but I would rather find a > framework, if such exists.
>> Can you share some more details on what you are trying to accomplish?
>> You have talked about tools (both of which I am unaware of) and said >> "batch", but apart from that there wasn't much more that could help anyone >> make an informed suggestion.
>> Let me know, I'll try to help :) >> Nuno
>> On Fri, Oct 5, 2012 at 11:08 AM, Guilherme Pim <pimgui...@gmail.com<javascript:> >> > wrote:
>>> Hi guys,
>>> Having the need to synchronize some data with external sources, I have >>> written some tasks on Grunt to do so. I wonder if Grunt is actually the >>> tool to write batch jobs in, or there actually is some other alternative.
>>> I have come across Node.io, but I rather stick to Grunt in this case. Do >>> you guys know about some alternatives, please?
Sounds like you just want a node service that does the scraping, and provides an http api for adding tasks and querying what tasks are currently running. Then have a separate command-line app that can talk to the web service. And maybe also an express app that can talk to it via a browser. I use that pattern a lot, but haven't yet written an abstraction for it. An abstraction for easily banging out command-line apps with tab-complete, etc that just talk to web services would be a useful thingie though.
I'm not sure if there's a generic node-based job-processor out there, but you might like https://github.com/defunkt/resque. It's ruby, but each job could just be a system call to start a node process.
Ted
On Oct 5, 2012, at 4:22 AM, Guilherme Pim <pimguilhe...@gmail.com> wrote:
> I am crawling some web pages to populate a database, and currently doing that with Grunt (http://gruntjs.com/), which is a task-based 'build tool' (that gets me confused, is it supposed to solely build?).
> So, I have created some scripts in the following manner:
> ========== Task file ==========
> module.exports = function (grunt) {
> grunt.registerTask('courses/update-info', 'Updates all courses info on the database', ... )
> }
> ============================
> So then I may have a somewhat controlled access to these jobs from the terminal, with ability to list all existing jobs and their descriptions. Also, I may pass some parameters to each task, although it's not very readable because they can't be associated with a name:
> grunt courses/update-info:4:5:2 (these will be passed as the tasks's function arguments)
> I guess I could go about creating node scripts and using a parameters library along with console colors and stuff, but I would rather find a framework, if such exists.
> Did I make myself clearer at all? haha
> Thanks for you time,
> Guilherme
> 2012/10/5 Nuno Job <nunojobpi...@gmail.com>
> Can you share some more details on what you are trying to accomplish?
> You have talked about tools (both of which I am unaware of) and said "batch", but apart from that there wasn't much more that could help anyone make an informed suggestion.
> Let me know, I'll try to help :)
> Nuno
> On Fri, Oct 5, 2012 at 11:08 AM, Guilherme Pim <pimguilhe...@gmail.com> wrote:
> Hi guys,
> Having the need to synchronize some data with external sources, I have written some tasks on Grunt to do so. I wonder if Grunt is actually the tool to write batch jobs in, or there actually is some other alternative.
> I have come across Node.io, but I rather stick to Grunt in this case. Do you guys know about some alternatives, please?
> Sounds like you just want a node service that does the scraping, and provides an http api for adding tasks and querying what tasks are currently running. Then have a separate command-line app that can talk to the web service. And maybe also an express app that can talk to it via a browser. I use that pattern a lot, but haven't yet written an abstraction for it. An abstraction for easily banging out command-line apps with tab-complete, etc that just talk to web services would be a useful thingie though.
> I'm not sure if there's a generic node-based job-processor out there, but you might like https://github.com/defunkt/resque. It's ruby, but each job could just be a system call to start a node process.
> Ted
> On Oct 5, 2012, at 4:22 AM, Guilherme Pim <pimguilhe...@gmail.com> wrote:
>> Okay, my bad!
>> I am crawling some web pages to populate a database, and currently doing that with Grunt (http://gruntjs.com/), which is a task-based 'build tool' (that gets me confused, is it supposed to solely build?).
>> So, I have created some scripts in the following manner:
>> ========== Task file ==========
>> module.exports = function (grunt) {
>> grunt.registerTask('courses/update-info', 'Updates all courses info on the database', ... )
>> }
>> ============================
>> So then I may have a somewhat controlled access to these jobs from the terminal, with ability to list all existing jobs and their descriptions. Also, I may pass some parameters to each task, although it's not very readable because they can't be associated with a name:
>> grunt courses/update-info:4:5:2 (these will be passed as the tasks's function arguments)
>> I guess I could go about creating node scripts and using a parameters library along with console colors and stuff, but I would rather find a framework, if such exists.
>> Did I make myself clearer at all? haha
>> Thanks for you time,
>> Guilherme
>> 2012/10/5 Nuno Job <nunojobpi...@gmail.com>
>> Can you share some more details on what you are trying to accomplish?
>> You have talked about tools (both of which I am unaware of) and said "batch", but apart from that there wasn't much more that could help anyone make an informed suggestion.
>> Let me know, I'll try to help :)
>> Nuno
>> On Fri, Oct 5, 2012 at 11:08 AM, Guilherme Pim <pimguilhe...@gmail.com> wrote:
>> Hi guys,
>> Having the need to synchronize some data with external sources, I have written some tasks on Grunt to do so. I wonder if Grunt is actually the tool to write batch jobs in, or there actually is some other alternative.
>> I have come across Node.io, but I rather stick to Grunt in this case. Do you guys know about some alternatives, please?
