I need to develop a piece of code which will run on Windows machines, doing some bidirectional RPC with a central server. I won't need any user interface, the software will run as a service. Since I don't have much experience developing on such platform, I'm seriously considering to use Common Lisp for such task.
This is my question: can you recommend me any Common Lisp implementation for Windows that can assist me in this task. A plus would be a link to some documentation. I'd rather use a free implementation, but a commercial one is also ok.
After some googling I found that Franz has something about it, I downloaded id and will give it a try. Is there some other way to accomplish.
> I need to develop a piece of code which will run on Windows machines, doing > some bidirectional RPC with a central server. I won't need any user > interface, the software will run as a service. Since I don't have much > experience developing on such platform, I'm seriously considering to use > Common Lisp for such task.
> This is my question: can you recommend me any Common Lisp implementation for > Windows that can assist me in this task. A plus would be a link to some > documentation. I'd rather use a free implementation, but a commercial one is > also ok.
> After some googling I found that Franz has something about it, I downloaded > id and will give it a try. Is there some other way to accomplish.
It depends on your needs. A stand alone (command line) application that will be executed using the Windows Scheduling Tasks service (under any user that you like) may be enough.
I don't see why people usually choose the more complicated "Windows Services" way instead of writing a simple program. The only concern is that this service must run without any users logged on and should stay "hidden" (no annoying console/gui window). You can accomplish this using the solution mentioned above.
blandest wrote: > On Jan 31, 12:21 am, Francisco Vides =??B?RmVybsOhbmRleg==?= > <fvi...@dedaloingenieros.com> wrote: >> Dear Lispers
>> I need to develop a piece of code which will run on Windows machines, >> doing some bidirectional RPC with a central server. I won't need any user >> interface, the software will run as a service. Since I don't have much >> experience developing on such platform, I'm seriously considering to use >> Common Lisp for such task. > It depends on your needs. A stand alone (command line) application > that will be executed using the Windows Scheduling Tasks service > (under any user that you like) may be enough.
> I don't see why people usually choose the more complicated "Windows > Services" way instead of writing a simple program. The only concern is > that this service must run without any users logged on and should stay > "hidden" (no annoying console/gui window). You can accomplish this > using the solution mentioned above.
Excuse my ignorance on windows (I'm a long time Linux user) but Windows Scheduling sounds more like cron, it will execute the program at certain times. I need a process started when the machine boots (I don't care if there's an user logged or not) and keeps executing in background until the machine is powered off. So it looks me like a service. Am I wrong?
And, again, can I do it with some Common Lisp implementation (other than Allegro, which I've already found)
Francisco Vides =??B?RmVybsOhbmRleg==?= <fvi...@dedaloingenieros.com> writes:
> blandest wrote:
>> On Jan 31, 12:21 am, Francisco Vides =??B?RmVybsOhbmRleg==?= >> <fvi...@dedaloingenieros.com> wrote: >>> Dear Lispers
>>> I need to develop a piece of code which will run on Windows machines, >>> doing some bidirectional RPC with a central server. I won't need any user >>> interface, the software will run as a service. Since I don't have much >>> experience developing on such platform, I'm seriously considering to use >>> Common Lisp for such task. > [...] > And, again, can I do it with some Common Lisp implementation (other than > Allegro, which I've already found)
But if you need to do bidirectional RPC at the same time, you may need a multi-threaded implementation, which case it's a little too soon for clisp. But I'm not sure sbcl can do threads on MS-Windows yet either.
<fvi...@dedaloingenieros.com> wrote: > blandest wrote: > > On Jan 31, 12:21 am, Francisco Vides =??B?RmVybsOhbmRleg==?= > > <fvi...@dedaloingenieros.com> wrote: > >> Dear Lispers
> >> I need to develop a piece of code which will run on Windows machines, > >> doing some bidirectional RPC with a central server. I won't need any user > >> interface, the software will run as a service. Since I don't have much > >> experience developing on such platform, I'm seriously considering to use > >> Common Lisp for such task. > > It depends on your needs. A stand alone (command line) application > > that will be executed using the Windows Scheduling Tasks service > > (under any user that you like) may be enough.
> > I don't see why people usually choose the more complicated "Windows > > Services" way instead of writing a simple program. The only concern is > > that this service must run without any users logged on and should stay > > "hidden" (no annoying console/gui window). You can accomplish this > > using the solution mentioned above.
> Excuse my ignorance on windows (I'm a long time Linux user) but Windows > Scheduling sounds more like cron, it will execute the program at certain > times.
Yes, but it will execute on boot too (see schtasks.exe on Windows XP/ 2003).
> I need a process started when the machine boots (I don't care if > there's an user logged or not) and keeps executing in background until the > machine is powered off. So it looks me like a service. Am I wrong?
The problem with Windows services is that they are harder to maintain than a simple console application. I've seen the Allegro API and it looks a little bit too complicated for what you need (but maybe I've misunderstood your intention ...). At work, my team is currently maintaining such a service (written in C# .Net). We don't send custom control commands to this service (only the standard start/stop). The advantage of simple application is that you can easily test it using the command line (and your user account). As far as I know that is impossible with a Windows Service. Also, you won't depend on some specific implementation that happens to implement the required API (such as Allegro).
Bourguignon) wrote: >Francisco Vides =??B?RmVybsOhbmRleg==?= <fvi...@dedaloingenieros.com> writes:
>> blandest wrote:
>>> On Jan 31, 12:21 am, Francisco Vides =??B?RmVybsOhbmRleg==?= >>> <fvi...@dedaloingenieros.com> wrote: >>>> Dear Lispers
>>>> I need to develop a piece of code which will run on Windows machines, >>>> doing some bidirectional RPC with a central server. I won't need any user >>>> interface, the software will run as a service. Since I don't have much >>>> experience developing on such platform, I'm seriously considering to use >>>> Common Lisp for such task. >> [...] >> And, again, can I do it with some Common Lisp implementation (other than >> Allegro, which I've already found)
>But if you need to do bidirectional RPC at the same time, you may need >a multi-threaded implementation, which case it's a little too soon for >clisp. But I'm not sure sbcl can do threads on MS-Windows yet either.
It's more than that. A Windows service that does any significant processing usually must be multithreaded because there are time limits for responding to service manager requests - if the program is busy and doesn't reply, the service manager may kill it.
That said, a lot can be done with a single thread and non-blocking I/O. If the problem can be solved by overlapped I/O, a single thread may be fine.
If the OP has access to Visual Studio, it can generate the framework of a .NET service in C#. There are also native service program frameworks in C and C++ buried in the Windows SDK. The OP could then embed ECL or Guile or whatever in the framework. That would solve the threading problem if necessary - the thread could be started by the framework and run the embedded Lisp.
blandest wrote: >> Excuse my ignorance on windows (I'm a long time Linux user) but Windows >> Scheduling sounds more like cron, it will execute the program at certain >> times.
> Yes, but it will execute on boot too (see schtasks.exe on Windows XP/ > 2003).
Then this could be the way to go. Many thanks for your help!
George Neuner wrote: >>But if you need to do bidirectional RPC at the same time, you may need >>a multi-threaded implementation, which case it's a little too soon for >>clisp. But I'm not sure sbcl can do threads on MS-Windows yet either.
> It's more than that. A Windows service that does any significant > processing usually must be multithreaded because there are time limits > for responding to service manager requests - if the program is busy > and doesn't reply, the service manager may kill it.
I will need multithreading since there could me simultaneous incoming requests.
> That said, a lot can be done with a single thread and non-blocking > I/O. If the problem can be solved by overlapped I/O, a single thread > may be fine.
> If the OP has access to Visual Studio, it can generate the framework > of a .NET service in C#. There are also native service program > frameworks in C and C++ buried in the Windows SDK. The OP could then > embed ECL or Guile or whatever in the framework. That would solve the > threading problem if necessary - the thread could be started by the > framework and run the embedded Lisp.
<fvi...@dedaloingenieros.com> writes: > I will need multithreading since there could me simultaneous incoming > requests.
Just a FYI, you don't. You can use non-blocking I/O to accept connections, by using the backlogging features of the sockets implementation. You should use that anyway.
Of course, more connections than the backlog allows will always be a problem, even if you multithread.
>>>But if you need to do bidirectional RPC at the same time, you may need >>>a multi-threaded implementation, which case it's a little too soon for >>>clisp. But I'm not sure sbcl can do threads on MS-Windows yet either.
>> It's more than that. A Windows service that does any significant >> processing usually must be multithreaded because there are time limits >> for responding to service manager requests - if the program is busy >> and doesn't reply, the service manager may kill it.
>I will need multithreading since there could me simultaneous incoming >requests.
Simultaneous I/O requests are not necessarily a reason for multithreading ... whether you need threads depends more on how many requests you expect and how much CPU work must be done to service each one. With asynchronous I/O, the I/O itself doesn't count as work.
Do you know about the 'select' function in _nix? Do you know how to handle multiple sockets (or files) using it?
Windows calls its asynchronous I/O "overlapped" and it works a bit differently from 'select', but the result is the same ... you start an I/O operation and go do something else while it happens. Windows tells you it's finished by signaling a user provided event (per I/O call if you like).
You can do quite a lot with asynchronous I/O and a single thread. It's usually relatively easy to program a state model by breaking up non-trivial request processing into explicit steps, tracking the progress of each request, and looping through the active requests performing the next step when necessary.
You may think "why do that when I have real OS threads"? Well, OS threads have not inconsiderable overhead, both in scheduling cycles and memory. For high activity server processes, it is frequently more efficient to handle all the I/O with one thread and use pool threads if necessary for heavy computing tasks. You can always add a new I/O thread if the existing ones are saturated. If you go the thread-per-request route, a heavily loaded server runs the risk of hitting process or system limits, running out of address space (at least on 32-bit), thrashing, etc. In general, with a server process, I think it's preferable to keep the number of OS threads involved to a minimum. [I'm sure someone else here will tell you the exact opposite. Where people stand on this issue depends on their background ... mine is operating systems, embedded, real time and high availability (24/7, uptime in months) applications. Lots of web programmers see no wrong in using OS threads. I say "use them sparingly". I do support "green" threading (user space threads) as an alternative to explicit state models if a decent package is available. I haven't been very impressed with green threading in any Lisp I've yet seen.]
Anyway, the normal model of a Windows service is a main thread that responds to Service Manager requests and one or more worker threads that do the actual job of the program. All the threads need to be coordinated so the Service Manager is aware of the program's status and can control it. I haven't seen what Lispworks or Allegro have in the way of Window service frameworks, but if you embed some Lisp into a C based service framework I think you'll be better off keeping the number of threads to a minimum.
George Neuner <gneun...@comcast.net> writes: > In general, with a server process, I think it's preferable to keep > the number of OS threads involved to a minimum.
I think this is generally correct but a useful extension of the idea is to pick the number of your threads in some ratio to the number of CPUs on the box. E.g. if you have an 8 cpu box, then 1 thread is leaving most of the box idle. Also >1000 threads is likely not optimal, due to scheduling overhead.
Probably 8-16 threads is optimal. Profile, profile, profile, then choose.
<udode...@users.sourceforge.net> wrote: > GN> Do you know about the 'select' function in _nix? Do you know how to > GN> handle multiple sockets (or files) using it?
>Windows has `select` too
Yes, but Windows select() _only_ works with sockets. _nix select() also works with files.
You can mix using sockets and files (and pipes and mailboxes) in Windows but you have to do it with WaitMultipleEvent().
>> In general, with a server process, I think it's preferable to keep >> the number of OS threads involved to a minimum.
>I think this is generally correct but a useful extension of the idea >is to pick the number of your threads in some ratio to the number of >CPUs on the box. E.g. if you have an 8 cpu box, then 1 thread is >leaving most of the box idle. Also >1000 threads is likely not >optimal, due to scheduling overhead.
>Probably 8-16 threads is optimal. Profile, profile, profile, then >choose.
>-russ
I agree. My response to the OP was kind of knee-jerk ... but a lot of people reach for threads as a first resort rather than analyzing the problem and deciding whether (and how many) threads are really needed.
Russell McManus <russell_mcma...@yahoo.com> writes: > E.g. if you have an 8 cpu box, then 1 thread is leaving most of the > box idle.
Rather than `idle', I'd say `free for other processes'. If you're expecting that your server is the only thing running on the machine, and you can make use of the extra processors then go for it.
George Neuner wrote: > I'm sure someone else here will tell you the exact opposite. Where > people stand on this issue depends on their background ... mine is > operating systems, embedded, real time and high availability (24/7, > uptime in months) applications. Lots of web programmers see no wrong > in using OS threads.
Yeah! Your answer made some os-related-long-unused neural connections blink again, and I thank you for that (good food for thought) :)
That said, the problem I'm solving: * won't have a high number of requests * each request will simply add a new job to a queue to be resolved by an human agent, and this will take a long time (hours, maybe days), so high response rate is not a problem.
So, to keep things simple, I'll stick to the threads based solution.
Of course I understand that with other kind of requirements (real time, for example) I would be forced to use other kind of solutions.
On Jan 31, 10:15 pm, Francisco Vides =??B?RmVybsOhbmRleg==?=
<fvi...@dedaloingenieros.com> wrote: > I will need multithreading since there could me simultaneous incoming > requests.
In this case, you have too little to choose from. Only two CL implementations run multi threading, OpenMCL and SBCL.
Only one is available on Win32, it is SBCL, but SBCL on Win32 still does NOT support unix-like SELECT, even then it is supported in Win32 for TCP sockets.
So I would recommend thinking twice before going this way. On Unix/linux, LISP is a king, on Win32 things are not that good. I myself had to switch to linux only to continue development in lisp.
On 2009-02-02 19:55:49 -0500, Zach Beane <x...@xach.com> said:
> Not true. Corman Lisp, Allegro Common Lisp, and LispWorks Common Lisp > all support threads on Windows.
Right, but with LispWorks for example only one thread can be running in the lisp at a time. Others that are pure foreign calls might run concurrently but you can't have two *lisp* threads running concurrently on a multi core machine. So even though LispWorks uses native threads, those native threads won't use multiple cores if those threads are running lisp code. -- Raffael Cavallaro, Ph.D.
> > Not true. Corman Lisp, Allegro Common Lisp, and LispWorks Common Lisp > > all support threads on Windows.
> Right, but with LispWorks for example only one thread can be running in > the lisp at a time. Others that are pure foreign calls might run > concurrently but you can't have two *lisp* threads running concurrently > on a multi core machine. So even though LispWorks uses native threads, > those native threads won't use multiple cores if those threads are > running lisp code.
I guess I should just dig in lisp-hug, but isn't this LW limitation only for MacOS?
>> Not true. Corman Lisp, Allegro Common Lisp, and LispWorks Common Lisp >> all support threads on Windows.
> Right, but with LispWorks for example only one thread can be running in > the lisp at a time. Others that are pure foreign calls might run > concurrently but you can't have two *lisp* threads running concurrently > on a multi core machine. So even though LispWorks uses native threads, > those native threads won't use multiple cores if those threads are > running lisp code.
LispWorks 6.0 will not only have removed that limitation, but will actually introduce support for running threads on different cores.
"16.5.1 Native threads on Windows, Mac OS X, Linux and FreeBSD Each Lisp mp:process has a separate native thread. You can have many runnable mp:process objects/native threads, but Lisp code can only run in one thread at a time and a lock is used to enforce this. This can limit performance on multi-CPU machines."
|
