Dealing with marketing types...
flyingfred0
had time) has been using (wx)Python/PostgreSQL for over 2 years and
developed a successful 1.0 release of a client/server product.
A marketing/product manager has brought in additional management and
"architecture" experts to propose moving the entire thing to a Java
(application server) platform for the next release. They want a
"scalable, enterprise solution" (though they don't really know what that
means) and are going crazy throwing around the Java buzzwords (not to
mention XML).
The developers (including myself) are growing uneasy; the management is
continuing to push their requirements and ignore the engineers. I think
there's still hope, but I'm at a loss for ideas beyond pointing out the
success stories of Python and Zope and language comparisons between
Python and Java.
What experiences have those in the Python community had in these kinds
of situations?
EP
> A small software team (developers, leads and even the manager when he's
> had time) has been using (wx)Python/PostgreSQL for over 2 years and
> developed a successful 1.0 release of a client/server product.
>
> A marketing/product manager has brought in additional management and
> "architecture" experts to propose moving the entire thing to a Java
> (application server) platform for the next release. They want a
> "scalable, enterprise solution" (though they don't really know what
> that means) and are going crazy throwing around the Java buzzwords (not to
> mention XML).
Marketing needs a compelling story to tell the customer: why _this_ technology? Java and XML are cheap buzz words to throw around (but not too much buzz left anymore!) What they need is a story, not a bunch of buzz words, but that story needs to fit into the customers' world view, it needs to mean something to the customer.
It's possible that Python/PostgreSQL is a technology combination that represents a winning story, at least in the right marketplaces. Possibly the development team is more technically savvy than the marketers ;-), so try to gently help them understand why the Python/PostgreSQL story is strong.
Faster incremental development of releases ("Customer, no long waits for the new features you want like you have with Java apps. Ou development is fast and agile like your requirements!") might be the kind of perspective that helps your cause. Give the marketing guys "stories" about why the current product implementation is good from a marketing perspective - how (wx)Python/PostgreSQL will make the product unique and noticeable and easier to sell than another Java/XML client server app.
I think that's about the best you can do.
good luck!
Eric
(Psuedo-marketing-type)
Will McGugan
Marketing types need a bandwagon to jump on. Point out that Google is
used by Google, ILM and NASA.
Will McGugan
--
http://www.willmcgugan.com
"".join({'*':'@','^':'.'}.get(c,0) or chr(97+(ord(c)-84)%26) for c in
"jvyy*jvyyzpthtna^pbz")
Diez B. Roggisch
> Marketing types need a bandwagon to jump on. Point out that Google is
> used by Google, ILM and NASA.
Certainly a true statement - but I've got the sneaky suspicion that the
first google was supposed to be python.
Diez
Harald Massa
> "scalable, enterprise solution" (though they don't really know what
> that means) and are going crazy throwing around the Java buzzwords
> (not to mention XML).
>
You can google it up, "astronaut architects"
There is a cheap solution: on this years EuroPython (www.europython.org)
there will be a special Slot in Social Skills track dealing with
"Selling Python", giving you a Python Sales Pitch and two more excellent
seminars about persuading people. More than that, in Python in Business
Track we will do slots about using Python for real worthy enterprise
apps which scale and are FULLY buzzword-compatible.
Join us!
Harald Armin Massa
GHUM Harald Massa
perusasion. python. postgresql.
phil
>
> What experiences have those in the Python community had in these kinds
> of situations?
>
developing software at home. In Python.
Java is a clumsy kludge. And the java environment has gone to hell.
Managers DO NOT listen to engineers. Marketing people cannot
spell indianere. Sorry, to be so pessimistic.
They are gonna outsource anyway. It's such a sexy buzzword,
how can they resist? How can you have an interesting
management/marketing meeting in which noone knows their
head from their richard, without cool buzzwords?
There are lots of good projects to work on at home.
Get your unemployment then a good waiter job.
Seriously, its cottage industry/small business that drives this
world, not the clowns you are describing. They are losers.
Don't let them drag you down.
If you really want to fight this on their terms, you need
engineering to write a white paper which is absolute horsedooey
but is loaded with stuff they cannot possible comprehend
and arrives at totally unsupported conclusions that you like.
Have all the engineers sign off and get some authoritative
endorsement. This will introduce fear and paralysis at
the mgmt level. Then get a couple happy customers and
lock it in.
fuzzylollipop
Thomas Bartkus
"flyingfred0" <flyin...@gmail.com> wrote in message
news:8B6qe.1742$pa3...@newsread2.news.atl.earthlink.net...
Sigh!
> The developers (including myself) are growing uneasy; the management
is
> continuing to push their requirements and ignore the engineers.
No - they are not pushing "requirements" here.
They are trying to specify the tools that must be used in order to achieve
those requirements. Sort of like me specifying the brand name and type of
tools the repair shop must use when they replace my alternator. Well - not
*quite* like that since I don't enjoy the power of a true employer/employee
relationship with my repair shop. But you get the picture.
It's a given that management has no way to reasonably evaluate on the
technical merits. However, there is one legitimate reason they might want
to do this. It is a non-technical yet nevertheless reasonable consideration.
Management needs to know they have a reliable labor pool to draw upon for
replacements. If that "small software team" decides to jump ship (or asks
for more $, or already makes enough $ to be attractive targets for
replacement) - would they be able hire the replacement expertise to carry
on? Management is *always* looking to lose the high priced creative
geniuses who brought them to the party. I know this from years as an
independent consultant talking to those managers. I can't tell you how many
times they were simply looking to replace the now highly paid guru(s) with
younger/lower cost and more recently - offshored labor.
That, my friend, is the real reason behind the "Java" buzzword. If that's
what your up against, I'm sorry to say that there are simply hoards of Java
underemployeds out there ready to flood human resources with resumes.
Worse - there are hoards of underbidding (lying? scumsucking?) contractors
with dozens of "Java" experts on the bench and no one with Python
experience. I gaurantee that the likes of these are tripping over one
another to get your employers attention.
If this is the case, then management throws up blather like "scalable" and
"enterprise solution" when they really mean they would like to reduce the
cost and increase the reliability of the labor force available to develop
and maintain the system.
If *thats* what's bothering you bunky - I'm sorry to tell you that I am
short on solutions.
BUT understanding the problem is the first step on the path to a solution
:-)
Thomas Bartkus
Thomas Bartkus
news:1118429907.1...@g43g2000cwa.googlegroups.com...
> get your resume in order and start looking . . .
I *hate* the fact that I agree with this post.
I, for one, am hoping for serious discussion to address the problem.
Thomas Bartkus
Kay Schluehr
> A small software team (developers, leads and even the manager when he's
> had time) has been using (wx)Python/PostgreSQL for over 2 years and
> developed a successful 1.0 release of a client/server product.
>
> A marketing/product manager has brought in additional management and
> "architecture" experts to propose moving the entire thing to a Java
> (application server) platform for the next release. They want a
> "scalable, enterprise solution" (though they don't really know what that
> means) and are going crazy throwing around the Java buzzwords (not to
> mention XML).
The product managers and technology responsibles ( marketing people do
less technology decisions at least in those german companys I know from
inside ) may not know which is the "best" technology on the market (
who really knows? ), but he clearly knows the discourse about
technology and he is right in not spending to much trusts in the
sensitivities, vanities and the pets of his developers ( Some like
dotNet, others like Java. Some believe that Perl is the hit and others
stick to "Ruby on Rails".)
> The developers (including myself) are growing uneasy; the management is
> continuing to push their requirements and ignore the engineers. I think
> there's still hope, but I'm at a loss for ideas beyond pointing out the
> success stories of Python and Zope and language comparisons between
> Python and Java.
The difference between J2EE and Zope may be that SUN promotes it's new
major releases half a year before they enter the market ( EJB3 ) while
Zope 3 ( Zope X3 to be accurate ) vanishes for half a year after it is
released ( if you read the docs e.g. the 'roadmap' in the Zope 3 wiki
you may have the impression the project is completely dead ).
> What experiences have those in the Python community had in these kinds
> of situations?
Python projects are submarines. You have to care not to go up to soon.
Kay
Roy Smith
> Python projects are submarines.
Sometimes submarines disappear without a trace and loss of all hands
aboard.
Will McGugan
Indeed. D'oh.
Terry Hancock
> > They want a
> > "scalable, enterprise solution" (though they don't really know what
> > that means) and are going crazy throwing around the Java buzzwords
> > (not to mention XML).
> >
> There is a very cheap solution: Ryan Tomayko debunkes all these myths.
> You can google it up, "astronaut architects"
Apparently not -- I can't find anything relevant on the first page with the
searches:
astronaut architects
"astronaut architects"
"astronaut architects" "ryan tomayko"
"astronaut architects" tomayko
"astronaut architects" tomko
"ryan tomayko"
However, after a bit of brainstorming, I tried:
"architecture astronauts" "ryan tomayko"
and got this:
http://naeblis.cx/rtomayko/2005/05/28/ibm-poop-heads
which is probably what you meant.
I love that file name. ;-)
Cheers,
Terry
--
Terry Hancock ( hancock at anansispaceworks.com )
Anansi Spaceworks http://www.anansispaceworks.com
Terry Hancock
> Python projects are submarines. You have to care not to go up to soon.
Ooh, I like that. I'm going to file that under "useful excuses".
Could come in handy! ;-D
fuzzylollipop
Joel Spolskyl was talking about
fuzzylollipop
already lost. I have been on both sides of this scenario, the "new
guys" were brought in and will win since they are the new "experts from
out of town". There may be some other _VALID_ business reason that
management has already made up their mind to hire these Java people.
Probably because they want to sell the company or merge with someone or
something and having a Java product would make them more attractive.
There are 2 things he can do.
1. Get your resume ready and approach the CEO or whomever and say. Why
is this happening? Since I can guarantee you they have already decided
to port this app to Java.
2. Be quiet, keep his head down, learn Java fasssstt, start agreeing
with the new party line and get on the bandwagon if he really wants to
stay at this company ( I wouldn't )
Kay Schluehr
> There are 2 things he can do.
>
> 1. Get your resume ready and approach the CEO or whomever and say. Why
> is this happening? Since I can guarantee you they have already decided
> to port this app to Java.
The CEO is probably indifferent about particular technology issues but
certainly not about money. Trashing a ready to use project ( it does
not seem to be a functional prototype ) is clearly a waste of effort
and time. Offer him to draw an architecture fulfilling the requirements
of the "enterprise solution" reusing existing components that is much
less expensive. Arguing that a Python project definitely needs less
programmers than the Java counterpart ( which is very cost effective
because you need less management and administration to lead them -
remember that a programmer is always cheap compared to a manager ). A
new ambitious manager wants to enforce changes. That's part of the game
and one has to assume this.
Remember also that there is not only a lot of hype and buzz around Java
technologies but one can also spread adjectives like "agile" in the
debate and buzz back ( doing politics ). There's still a lot to exploit
in this direction: the "agile" company/department relies on lightweight
methodologies, technologies, languages, thinking, eating, drinking and
pissing.
O.K. if the boss is nervous, carefull and weak and goes directed by
it's servants one may be off-chance.
Kay
Drazen Gemic
>
>>
>> What experiences have those in the Python community had in these kinds
>> of situations?
>>
> Ive had lots of experience updating my resume and
>
> developing software at home. In Python.
>
> Java is a clumsy kludge. And the java environment has gone to hell.
I am a freelance developer and I prefer to do job in Java over anything
else. I love Python, and it is my favourite language, but there are some
problems.
With Java I depend very little on customers IT staff, sysadmins, etc. If
I need additional functionality in form library, extension, whatever, all
I need is to drop another JAR in, and that does it.
With Python, the situation is different. Often, some kind of modules are
required to be compiled, or packages installed. Generaly, I do not have sufficient
privileges on customers system to do that. IT personell might be willing
and capable to do that, and might be not. There could be some
inter-department politics and/or friction involved at customers side
which may turn into serious obstacle.
Thnks, but I stick with Java. And considering Jython in the future.
DG
Aahz
flyingfred0 <flyin...@gmail.com> wrote:
>
>A small software team (developers, leads and even the manager when he's
>had time) has been using (wx)Python/PostgreSQL for over 2 years and
>developed a successful 1.0 release of a client/server product.
>
>A marketing/product manager has brought in additional management and
>"architecture" experts to propose moving the entire thing to a Java
>(application server) platform for the next release. They want a
>"scalable, enterprise solution" (though they don't really know what that
>means) and are going crazy throwing around the Java buzzwords (not to
>mention XML).
Point out that pushing this means they're almost certainly going to lose
at least some of their development team, which means the next release is
going to start from ground zero without the necessary expertise. Even
if that doesn't happen, switching to Java is going to take much more time
than improving the current product:
http://www.JoelOnSoftware.com/articles/fog0000000069.html
--
Aahz (aa...@pythoncraft.com) <*> http://www.pythoncraft.com/
f u cn rd ths, u cn gt a gd jb n nx prgrmmng.
tom
> I was completely serious, he is _NOT_ going to win this one. He has
> already lost. I have been on both sides of this scenario, the "new guys"
> were brought in and will win since they are the new "experts from out of
> town".
Not only do I take you seriously - I agree!
I also have been on both sides of this scenario although my take on it is
slightly different. It's not so much the "experts from out of town" as it
is the tendency to dump the guy(s) that brought them to the party.
The sequence goes like this:
1) When there is little or no money to be made, you start out with an
implied status as a partner. This means you work long + extra hours for
little pay on the promise that you will be rewarded when/if success comes.
2) Then the product gets out the door and it's more long hours with little
pay. Much massaging and tweaking and still little money incoming.
3) Success & money start to roll in. Now your status drops from partner to
hired hand. An overpaid hired hand at that. Now that the heavy lifting is
done, managment is wondering whether they need to actually reward the
guy(s) who brought them to the party. The rational parts of their brains
shut down while every fiber of their larcenous beings wants them to
believe they can now dispense with the high priced talent (you!) for some
low bucks commodity labor. There scads of outsourcing firms tripping over
one another to sell them the latter.
> There may be some other _VALID_ business reason that management has
> already made up their mind to hire these Java people. Probably because
> they want to sell the company or merge with someone or something and
> having a Java product would make them more attractive.
Yes, there is a possible _VALID_ reason. That would be the perception,
probably accurate, that a technology like Java will shelter them from
total dependency on some individual developer (you!). In other words,
there is a greater likelihood that they can find replacement talent should
they need it. Thats the optimistic view. More often it sinks to the
sleazy when they decide to stiff the original guys who did all the extra
work up front. If they can replace them, there will be no need to "pay
off" on the extra work they did up front.
I have had this happen to me as an employee. Later, as an outside
consultant, I was frequently disgusted to realize how many manager/owners
were merely seeking to avoid the payoff for the guys who went the extra
mile to give them a profitable product. Tis business in the USA, sad to
say.
> There are 2 things he can do.
>
> 1. Get your resume ready and approach the CEO or whomever and say. Why
> is this happening? Since I can guarantee you they have already decided
> to port this app to Java.
Resume ready is certainly wise and I concur with your gaurantee.
> 2. Be quiet, keep his head down, learn Java fasssstt, start agreeing
> with the new party line and get on the bandwagon if he really wants to
> stay at this company ( I wouldn't )
I disgree here. The party line is nothing but a cover. The goal is to
break the dependency on the guru(s) who did the developement or worse,
outright replacement. The likelihood of staying on is slim and will
become increasingly unpleasant unless the employer is so lacking in
concience as to fire him outright.
Let me add an Item #3 -
If you have some entrepeneurial savvy and can keep your emotions out of
it tou can simply tell them you have decided strike out on your own and
tell them that you will be available. They will be happy to hear you are
leaving and happier still to hear you can be available for backup.
Their goals and fears are addressed at the same time. AND there is a very
high possibility that they will *need* you at a later date for which you
can charge them dearly.
That last item #3 has actually worked for me with (2) prior employers.
I did have to eat my indignation and keep it friendly but it did pay off
in the end.
Thomas Bartkus
bruce
but.. if you are part of an initial project.. get in writing what your role
is!!!! if you're as a partner, get it in writing... if you're as a hired
gun.. get it in writing... if you can't get anything in writing.. then make
sure you have your own copies of any/all code that you're created/been a
part of...
in this day/age, you need to protect yourself....
and trust me, if your code is valuable, and you walk away with it because
you feel you've been treated in an unfair manner (assuimng no written
statement of your role), then let the other guy come after you... things get
resolved in a much more equitable manner when the concerned parties are
reasonably equal...
-bruce
tom
> i don't know what the original thread is/was...
>
> but.. if you are part of an initial project.. get in writing what your role
> is!!!! if you're as a partner, get it in writing... if you're as a hired
> gun.. get it in writing... if you can't get anything in writing.. then make
> sure you have your own copies of any/all code that you're created/been a
> part of...
>
> in this day/age, you need to protect yourself....
>
> and trust me, if your code is valuable, and you walk away with it because
> you feel you've been treated in an unfair manner (assuimng no written
> statement of your role), then let the other guy come after you... things get
> resolved in a much more equitable manner when the concerned parties are
> reasonably equal...
>
> -bruce
I agree completely and you do need to read the original post.
Thomas Bartkus
Stephen Kellett
<E...@zomething.com> writes
>> that means) and are going crazy throwing around the Java buzzwords (not to
>> mention XML).
Sounds like someone has read about AJAX and decided that is what is
next. They probably put 2 and 2 together and came up with 5 thinking the
J stands for Java rather than Javascript and that your sever end must be
Java. Well if they really want performance play the C++ (or assembler!)
trump card and watch them squirm :-)
I think you best approach is some serious education of upper management
on the benefits and pitfalls of each language, and of switching
languages. Also point out the huge costs:
o Total write-off of all development costs of V1.0.
o Total write off of all intellectual property assets of V1.0 (well if
you are building V2.0 on something else, you've put V1.0 in the bin with
zero re-use)
o Total slap in the face and moral-crusher to the development team and
support staff for V1.0. You will most likely see an exodus of talented
staff after the change, if it happens.
o Effectively starting from ground-zero, making the cost for
implementing V2.0 the entire development cost, rather than the
incremental cost for the jump to V2.0 from V1.0 using the existing
language.
The costs in human, timescale and financial terms for what these people
are proposing are huge. This company may not survive the change. If they
change you may want to consider the "abandon ship" approach and find a
more reliable place to devote you skills to.
Finally, read "The Peter Principle" and realise there are people like
these with their sights set on getting to the top of the greasy pole
without any consideration for the damage they cause to others. You need
to identify such people and steer clear of them (they generally do not
infect all companies).
All the best.
Stephen
--
Stephen Kellett
Object Media Limited http://www.objmedia.demon.co.uk/software.html
Computer Consultancy, Software Development
Windows C++, Java, Assembler, Performance Analysis, Troubleshooting
Terry Reedy
"Terry Hancock" <han...@anansispaceworks.com> wrote in message
news:200506102209....@anansispaceworks.com...
Thanks for digging this up. It solidified my understanding of why LAMP.
TJR
Paul Rubin
> > http://naeblis.cx/rtomayko/2005/05/28/ibm-poop-heads
> > which is probably what you meant.
> Thanks for digging this up. It solidified my understanding of why LAMP.
That article makes a lot of bogus claims and is full of hype. LAMP is
a nice way to throw a small site together without much fuss, sort of
like fancy xerox machines are a nice way to print a small-run
publication without much fuss. If you want to do something big, you
still need an actual printing press. Hint: Google.com is not a LAMP
site despite that article's insinuation. Yes, Google uses Python for
lots of internal projects and maybe administrative pages. The search
engine is not a Python app.
Philippe C. Martin
cycle of many companies: R&D driven versus Marketing driver.
My belief is that none work as the trades do not attempt to reach the same
goal:
1) R&D should not try to define products
2) Marketing should not try to impose the tools/means necessary to implement
the products they have defined.
I know that does not help
Drazen Gemic
> The sequence goes like this:
> 1) When there is little or no money to be made, you start out with an
> implied status as a partner. This means you work long + extra hours for
> little pay on the promise that you will be rewarded when/if success comes.
Well, customers are NOT partners. You need a contract when you deal with
customers. It is not a team work, it is selling and buying. And you are
selling. This is a third year that I am working as a freelance, and,
during unofficial conversations with customers representatives,
I have heard many phrases like:
"I am your ally", "covering your back", "we are the team", and I never fell
for any of them. I agreed in general, never been impolite, but went on my
way. And I never forgot, neither let customers forget, that I am selling, and they are
buying, and we are not the team.
DG
Andrew Dalke
> That article makes a lot of bogus claims and is full of hype. LAMP is
> a nice way to throw a small site together without much fuss, sort of
> like fancy xerox machines are a nice way to print a small-run
> publication without much fuss. If you want to do something big, you
> still need an actual printing press.
In the comments the author does say he's trying to be provocative.
My question to you is - what is "something big"? I've not been
on any project for which "LAMP" can't be used, and nor do I
expect to be. After all, there's only about 100,000 people in
the world who might possibly interested using my software. (Well,
the software I get paid to do; not, say, the couple of patches I've
sent in to Python).
I had one client consider moving from Python/CGI/flat files to
Java/WebLogic/Oracle. The old code took nearly 10 seconds to
display a page (!). They were convinced that they had gone past
the point where Python/CGI was useful, and they needed to use a
more scalable enterprise solution. The conviction meant they
didn't profile the system. With about a day of work I got the
performance down to under a second by removing some needless imports,
delaying others until they were needed, making sure all the
.pyc files existed, etc.
I could have gotten more performance switching to a persistent
Python web server and using a database instead of a bunch of
flat files in a directory, but that wasn't worth the time.
Andrew
da...@dalkescientific.com
Paul Rubin
> My question to you is - what is "something big"? I've not been
> on any project for which "LAMP" can't be used, and nor do I
> expect to be. After all, there's only about 100,000 people in
> the world who might possibly interested using my software. (Well,
> the software I get paid to do; not, say, the couple of patches I've
> sent in to Python).
If you're running a web site with 100k users (about 1/3 of the size of
Slashdot) that begins to be the range where I'd say LAMP starts
running out of gas. Yes, Slashdot is a LAMP site, but it's split
across a rack full of servers and is spending kilobucks a month on
colo space and hosting fees. Other similarly sized sites face similar
expenses. It seems to me that by using implementation methods that
map more directly onto the hardware, a site with Slashdot's traffic
levels could run on a single modest PC (maybe a laptop). I believe
LiveJournal (which has something more like a million users) uses
methods like that, as does ezboard. There was a thread about it here
a year or so ago.
As a simple example, that article's advice of putting all fine grained
session state into the database (so that every single browser hit sets
off SQL queries) is crazy. One site I worked on got a huge speedup by
simply storing the most frequently used stuff from the user session in
a browser cookie. That required zero extra work to handle multiple
servers (whichever server got the query, got the cookie) and it saved
a ton of SQL traffic.
As for "big", hmm, I'd say as production web sites go, 100k users is
medium sized, Slashdot is "largish", Ebay is "big", Google is huge.
Andrew Dalke
> If you're running a web site with 100k users (about 1/3 of the size of
> Slashdot) that begins to be the range where I'd say LAMP starts
> running out of gas.
Let me elaborate a bit. That claim of 100K from me is the
entire population of people who would use bioinformatics or
chemical informatics. It's the extreme upper bound of the
capacity I ever expect. It's much more likely I'll only
need to handle a few thousand users.
> I believe
> LiveJournal (which has something more like a million users) uses
> methods like that, as does ezboard. There was a thread about it here
> a year or so ago.
I know little about it, though I read at
http://goathack.livejournal.org/docs.html
] LiveJournal source is lots of Perl mixed up with lots of MySQL
I found more details at
http://jeremy.zawodny.com/blog/archives/001866.html
It's a bunch of things - Perl, C, MySQL-InnoDB, MyISAM, Akamai,
memcached. The linked slides say "lots of MySQL usage."
60 servers.
I don't see that example as validating your statement that
LAMP doesn't scale for mega-numbers of hits any better than
whatever you might call "printing press" systems.
> As a simple example, that article's advice of putting all fine grained
> session state into the database (so that every single browser hit sets
> off SQL queries) is crazy.
To be fair, it does say "database plus cache" though the author
suggests the place for the cache is at the HTTP level and not
at the DB level. I would have considered something like memcached
perhaps backed by an asychronous write to a db if you want the
user state saved even after the cache is cleared/reset.
How permanent though does the history need to be? Your
approach wipes history when the user clears the cookie and it
might not be obvious that doing so should clear the history.
In any case, the implementation cost for this is likely
higher than what you did. I mention it to suggest an
alternative.
> As for "big", hmm, I'd say as production web sites go, 100k users is
> medium sized, Slashdot is "largish", Ebay is "big", Google is huge.
I'ld say that few sites have >100k users, much less
daily users with personalized information. As a totally made-up
number, only few dozens of sites (maybe a couple hundred?) would
need to worry about those issues.
If that's indeed the case then I'll also argue that each of
them is going to have app-specific choke points which are best
hand-optimized and not framework optimized. Is there enough
real-world experience to design a EnterpriseWeb-o-Rama (your
"printing press") which can handle those examples you gave
any better than starting off with a LAMP system and hand-caching
the parts that need it?
Andrew
da...@dalkescientific.com
Paul Rubin
> I know little about it, though I read at
> http://goathack.livejournal.org/docs.html
> ] LiveJournal source is lots of Perl mixed up with lots of MySQL
>
> I found more details at
> http://jeremy.zawodny.com/blog/archives/001866.html
>
> It's a bunch of things - Perl, C, MySQL-InnoDB, MyISAM, Akamai,
> memcached. The linked slides say "lots of MySQL usage." 60 servers.
LM uses MySQL extensively but what I don't know is whether it serves
up individual pages by the obvious bunch of queries like a smaller BBS
might. I have the impression that it's more carefully tuned than that.
> I don't see that example as validating your statement that
> LAMP doesn't scale for mega-numbers of hits any better than
> whatever you might call "printing press" systems.
What example? Slashdot? It uses way more hardware than it needs to,
at least ten servers and I think a lot more. If LJ is using 6x as
many servers and taking 20x (?) as much traffic as Slashdot, then LJ
is doing something more efficiently than Slashdot.
> How permanent though does the history need to be? Your
> approach wipes history when the user clears the cookie and it
> might not be obvious that doing so should clear the history.
The cookie is set at user login and it only has to persist through the
login session. It's not as if the info only exists in the cookie and
nowhere else.
> > As for "big", hmm, I'd say as production web sites go, 100k users is
> > medium sized, Slashdot is "largish", Ebay is "big", Google is huge.
>
> I'ld say that few sites have >100k users, much less
> daily users with personalized information. As a totally made-up
> number, only few dozens of sites (maybe a couple hundred?) would
> need to worry about those issues.
Yes, but for those of us interested in how big sites are put together,
those are the types of sites we have to think about ;-). I'd say
there's more than a few hundred of them, but it's not like there's
millions. And some of them really can't afford to waste so much
hardware--look at the constant Wikipedia fundraising pitches for more
server iron because the Wikimedia software (PHP/MySQL, natch) can't
handle the load.
> If that's indeed the case then I'll also argue that each of
> them is going to have app-specific choke points which are best
> hand-optimized and not framework optimized. Is there enough
> real-world experience to design a EnterpriseWeb-o-Rama (your
> "printing press") which can handle those examples you gave
> any better than starting off with a LAMP system and hand-caching
> the parts that need it?
Yes, of course there is. Look at the mainframe transaction systems of
the 60's-70's-80's, for example. Look at Google. Then there's the
tons of experience we all have with LAMP systems. By putting some
effort into seeing where the resources in those things go, I believe
we can do a much better job. In particular, those sites like Slashdot
are really not update intensive in the normal database sense. They
can be handled almost entirely with some serial log files plus some
ram caching. At that point almost all the SQL overhead and a lot of
the context switching can go away.
Kay Schluehr
> With Java I depend very little on customers IT staff, sysadmins, etc. If
> I need additional functionality in form library, extension, whatever, all
> I need is to drop another JAR in, and that does it.
Maybe this is for you?
http://peak.telecommunity.com/DevCenter/PythonEggs
Kay
Mike Meyer
>> medium sized, Slashdot is "largish", Ebay is "big", Google is huge.
> I'ld say that few sites have >100k users, much less
> daily users with personalized information. As a totally made-up
> number, only few dozens of sites (maybe a couple hundred?) would
> need to worry about those issues.
I'd say quite a *lot* of sites have >100k users. A small client of
mine was a (now defunct .com) that was focused on "community
building". They had a user base of a couple of million people, and
you've probably never heard of The Park. They ran six servers,
thousands of simultaneous users, and it was all built on LAMP.
If you go looking for sites that offer the same kinds of things they
did - free web hosting, free web-based email, web-based chat,
calendering services, etc., you'll find a lot of such sites, and they
all probably have more than 100K users.
Of course, when you're talking about millions of web sites, a "few
sites" could be a a fairly large number of them.
An article I read recently made the point that I think you're trying
to make. The author argued that for most sites, scalability just
wasn't that big an issue. Web sites are cheap enough that they are
affordable to relatively small communities, and in many cases a
service that would bomb if they tried to go global with it would be a
big success in a small community. As such, he expects the web to be
dominated by sites that are really only of interest to a small
community. For those sites, LAMP will work just fine.
<mike
--
Mike Meyer <m...@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
Steve Jorgensen
...
>Let me add an Item #3 -
>If you have some entrepeneurial savvy and can keep your emotions out of
>it tou can simply tell them you have decided strike out on your own and
>tell them that you will be available. They will be happy to hear you are
>leaving and happier still to hear you can be available for backup.
>Their goals and fears are addressed at the same time. AND there is a very
>high possibility that they will *need* you at a later date for which you
>can charge them dearly.
>
>That last item #3 has actually worked for me with (2) prior employers.
>I did have to eat my indignation and keep it friendly but it did pay off
>in the end.
>Thomas Bartkus
I have to say that, although I have yet to write a line of Python code for
money, I've found that this concept basically works. When you realize that
your employer is cutting you out to save the cost of paying you, funny enough,
they'll be willing to -really- pay you as a consultant later when they get
stuck, and one or more paying customers are impacted. They also win't mind
figuring out how to have you come in after hours so it won't conflict with
your new day job if you have one. In my case, the work was in VB/VBA, but the
same principle should apply to any technology.
Do make sure that your contract with any new employer allows you to do at
least small amounts of moonlighting - they probably won't object. They will
insist that any moonlighting shall not compete with their line of business,
and you should agree to that stipulation.
Aahz
Paul Rubin <http://phr...@NOSPAM.invalid> wrote:
>
>What example? Slashdot? It uses way more hardware than it needs to,
>at least ten servers and I think a lot more. If LJ is using 6x as
>many servers and taking 20x (?) as much traffic as Slashdot, then LJ
>is doing something more efficiently than Slashdot.
So what? I think you're missing the real point of the article: using
LAMP scales *DOWN* in a way that enterprise systems don't. Getting your
first prototype up and running is far more important than sheer
scalability, and LAMP does have many mechanisms to obtain scalability
when it's needed.
Terry Reedy
"Paul Rubin" <"http://phr.cx"@NOSPAM.invalid> wrote in message
news:7xwtp09...@ruckus.brouhaha.com...
> Andrew Dalke <da...@dalkescientific.com> writes:
>> If that's indeed the case then I'll also argue that each of
>> them is going to have app-specific choke points which are best
>> hand-optimized and not framework optimized. Is there enough
>> real-world experience to design a EnterpriseWeb-o-Rama (your
>> "printing press") which can handle those examples you gave
>> any better than starting off with a LAMP system and hand-caching
>> the parts that need it?
>
> Yes, of course there is. Look at the mainframe transaction systems of
> the 60's-70's-80's, for example. Look at Google.
Based on what I've read, if we could look at Google, we would see 150,000
to 200,000 servers (about half bought with IPO money). We would see a
highly customized dynamic cluster computing infrastructure that can be
utilized with high-level (Python-like) commands. The need to throw
hundreds of machines at each web request strikes me as rather specialized,
though definitely not limited to search. So while not LAMP, I don't see it
as generic EWeboRama either.
Terry J. Reedy
bruce
i've never seen guesses that high.. i've seen somewhere as high as possible
100K... but the author stated that he was purely guessing...
-bruce
-----Original Message-----
From: python-list-bounces+bedouglas=earthl...@python.org
[mailto:python-list-bounces+bedouglas=earthl...@python.org]On Behalf
Of Terry Reedy
Sent: Sunday, June 12, 2005 9:48 AM
To: pytho...@python.org
Subject: Re: Dealing with marketing types...
Terry J. Reedy
Paul Rubin
> So what? I think you're missing the real point of the article: using
> LAMP scales *DOWN* in a way that enterprise systems don't. Getting your
> first prototype up and running is far more important than sheer
> scalability,
There comes a day when your first prototype can no longer handle the
traffic. The question then is how do you deal with that.
> and LAMP does have many mechanisms to obtain scalability when it's
> needed.
LAMP seems to have no solution other than scaling (i.e. blowing more
money on hardware and colo space). One really gets the impression
that a more thoughtful design could handle the traffic without needing
to scale the hardware.
Andrew Dalke
> Andrew Dalke <da...@dalkescientific.com> writes:
...
>> I found more details at
>> http://jeremy.zawodny.com/blog/archives/001866.html
>>
>> It's a bunch of things - Perl, C, MySQL-InnoDB, MyISAM, Akamai,
>> memcached. The linked slides say "lots of MySQL usage." 60 servers.
>
> LM uses MySQL extensively but what I don't know is whether it serves
> up individual pages by the obvious bunch of queries like a smaller BBS
> might. I have the impression that it's more carefully tuned than that.
The linked page links to a PDF describing the architecture.
The careful tuning comes in part from a high-performance caching
system - memcached.
>> I don't see that example as validating your statement that
>> LAMP doesn't scale for mega-numbers of hits any better than
>> whatever you might call "printing press" systems.
>
> What example? Slashdot?
Livejournal. You gave it as a counter example to the LAMP
architecture used by /.
] It seems to me that by using implementation methods that
] map more directly onto the hardware, a site with Slashdot's
] traffic levels could run on a single modest PC (maybe a laptop).
] I believe LiveJournal (which has something more like a million
] users) uses methods like that, as does ezboard.
Since LJ uses a (highly hand-tuned) LAMP architecture, it isn't
an effective counterexample.
> It uses way more hardware than it needs to,
> at least ten servers and I think a lot more. If LJ is using 6x as
> many servers and taking 20x (?) as much traffic as Slashdot, then LJ
> is doing something more efficiently than Slashdot.
I don't know where the 20x comes from. Registered users? I
read /. but haven't logged into it in 5+ years. I know I
hit a lot /. more often than I do LJ (there's only one diary
I follow there). The use is different as well; all people
hit one story / comments page, and the comments are ranked
based on reader-defined evaluations. LJ has no one journal
that gets anywhere as many hits and there is no ranking scheme.
>> I'ld say that few sites have >100k users, much less
>> daily users with personalized information. As a totally made-up
>> number, only few dozens of sites (maybe a couple hundred?) would
>> need to worry about those issues.
>
> Yes, but for those of us interested in how big sites are put together,
> those are the types of sites we have to think about ;-).
My apologies since I know this sounds snide, but then why didn't
you (re)read the LJ architecture overview I linked to above?
That sounds like something you would have been interested in
reading and would have directly provided information that
counters what you said in your followup.
The "ibm-poop-heads" article by Ryan Tomayko gives pointers to
several other large-scale LAMP-based web sites. You didn't
like the Google one. I checked a couple of the others:
IMDB -
http://www.findarticles.com/p/articles/mi_zdpcm/is_200408/ai_ziff130634
As you might expect, the site is now co-located with other Amazon.com
sites, served up from machines running Linux and Apache, but ironically,
most of the IMDb does not use a traditional database back end. Its
message boards are built on PostgreSQL, and certain parts of IMDb
Pro-including its advanced search-use MySQL, but most of the site is
built with good old Perl script.
del.icio.us
Took some digging but I found
http://lists.del.icio.us/pipermail/discuss/2004-November/001421.html
"The database gets corrupted because the machine gets power-cycled,
not through any fault of MySQL's."
The point is that LAMP systems do scale, both down and up. It's
a polemic against "architecture astronauts" who believe the only
way to handle large sites (and /., LJ, IMDB, and del.icio.us are
larger than all but a few sites) is with some spiffy "enterprise"
architecture framework.
> I'd say
> there's more than a few hundred of them, but it's not like there's
> millions. And some of them really can't afford to waste so much
> hardware--look at the constant Wikipedia fundraising pitches for more
> server iron because the Wikimedia software (PHP/MySQL, natch) can't
> handle the load.
Could that have, for example, bought EnterpriseWeb-O-Rama and done
any better/cheaper? Could they have even started the project
had they gone that route?
> Yes, of course there is [exprience in large-scale web apps].
> Look at the mainframe transaction systems of the 60's-70's-80's, for
> example. Look at Google.
For the mainframe apps you'll have to toss anything processed
in batch mode, like payrolls. What had the customization level
and scale comparable to 100K+ sites of today? ATMs? Stock trading?
Google is a one-off system. At present there's no other system
I know of - especially one with that many users - where a single
user request can trigger searches from hundreds of machines.
That's all custom software. Or should most servers implement
what is in essence a new distributed operating system just to
run a web site?
> Then there's the tons of experience we all have with LAMP systems. By
> putting some effort into seeing where the resources in those things go,
> I believe we can do a much better job. In particular, those sites like
> Slashdot are really not update intensive in the normal database sense.
> They can be handled almost entirely with some serial log files plus some
> ram caching. At that point almost all the SQL overhead and a lot of the
> context switching can go away.
Is /. an appropriate comparable? I get the idea that it hasn't
changed much in the last, say, 5 years and the user base hasn't
grown much either. What you propose requires programming effort.
If the system doesn't need work, if money in > money out (even with
expensive hardware), and if the extra work doesn't get much benefit,
then is it worthwhile to them to rearchitect the system?
Perhaps in a couple years it'll run on two machines (one as the
backup), with no change to the code, and simply because the
hardware is good enough and cheap enough.
Andrew
da...@dalkescientific.com
Aahz
Paul Rubin <http://phr...@NOSPAM.invalid> wrote:
>>
>> So what? I think you're missing the real point of the article: using
>> LAMP scales *DOWN* in a way that enterprise systems don't. Getting your
>> first prototype up and running is far more important than sheer
>> scalability,
>
>There comes a day when your first prototype can no longer handle the
>traffic. The question then is how do you deal with that.
Then you scale with hardware or do profiling and rewrite chunks,
whichever costs less FOR THE BUSINESS.
>> and LAMP does have many mechanisms to obtain scalability when it's
>> needed.
>
>LAMP seems to have no solution other than scaling (i.e. blowing more
>money on hardware and colo space). One really gets the impression
>that a more thoughtful design could handle the traffic without needing
>to scale the hardware.
Is there some reason you keep repeating yourself without actually paying
attention to other people?
Thomas Bartkus
news:cr3oa11uutgn2et58...@4ax.com...
> On Sat, 11 Jun 2005 11:51:02 -0500, tom <thomas...@comcast.net> wrote:
>
> ...
> >Let me add an Item #3 -
> >If you have some entrepeneurial savvy and can keep your emotions out of
> >it tou can simply tell them you have decided strike out on your own and
> >tell them that you will be available. They will be happy to hear you are
> >leaving and happier still to hear you can be available for backup.
> >Their goals and fears are addressed at the same time. AND there is a
very
> >high possibility that they will *need* you at a later date for which you
> >can charge them dearly.
> >
> >That last item #3 has actually worked for me with (2) prior employers.
> >I did have to eat my indignation and keep it friendly but it did pay off
> >in the end.
> >Thomas Bartkus
>
> I have to say that, although I have yet to write a line of Python code for
> money, I've found that this concept basically works. When you realize
that
> your employer is cutting you out to save the cost of paying you, funny
enough,
> they'll be willing to -really- pay you as a consultant later when they get
> stuck, and one or more paying customers are impacted.
Yup! It's theold stupid + greedy double whammy that means they end up paying
more.
Your not feeling sorry for them, are you?
> They also win't mind
> figuring out how to have you come in after hours so it won't conflict with
> your new day job if you have one. In my case, the work was in VB/VBA, but
the
> same principle should apply to any technology.
>
> Do make sure that your contract with any new employer allows you to do at
> least small amounts of moonlighting - they probably won't object. They
will
> insist that any moonlighting shall not compete with their line of
business,
> and you should agree to that stipulation.
How much of *my* time are they buying with a salary? 40Hrs a week? 24/7 ?
You want to see that your contract as an employee doesn't somehow forbid you
from earning extra on your own time. Unless, of course, they are paying
enough to make you happy to sell them *all* your time. Sometimes you are
hired mainly to keep your skills unavailable to their competitors. Thats ok
as long as they pay you enough to keep you happy with this. Unless they are
paying for it, your own free time is none of their business.
Thomas Bartkus
Terry Hancock
You might be interested to know that California state law forbids anti-moonlighting
clauses, provided that no company resources are used by the employee in the
conduct of their own business (which means of course, you'd better not take
your business calls at work).
Not sure how many other jurisdictions have implemented something like
this, but it sounds like a very good thing to me.
--
Terry Hancock ( hancock at anansispaceworks.com )
Anansi Spaceworks http://www.anansispaceworks.com
fuzzylollipop
was 100% the DotCom plan, offer crap salary and tonnes of "options"
then fire/replace all the people that worked for _FREE_ practically
when the next round of funding comes in, rinse, repeat.
Either way . . . I think the guy needs to move on. I know I would.
fuzzylollipop
with code" someone else paid you to write. Regardless of what your
perceived slight is.
NEVER take code you were paid to write unless you have it in writing
that you can, you will lose that one everytime.
Thomas Bartkus
news:1118686670....@g14g2000cwa.googlegroups.com...
I'll agree here. If you were hired as an employee for salary or an hourly
wage, you will have no standing whatsover as "owner" of the work you
produced. If your ex employer charges you with theft, it will be a slam
dunk in court. You don't own what you produced for your employer. Even if
you were fool enough to do much if it on your own unpaid off hours. Even if
they reneged on some verbal promises to you in order to con you into doing
so. It sucks but that's the way it is.
However - I certainly sympathize with the desire to fight thievery with
thievery.
Unfortunately, you can't win that way. Unless, of course, you can do it
without being caught :-)
But that only gets you revenge - not payment.
Thomas Barkus
Cameron Laird
Kay Schluehr <kay.sc...@gmx.net> wrote:
.
.
.
>less expensive. Arguing that a Python project definitely needs less
>programmers than the Java counterpart ( which is very cost effective
>because you need less management and administration to lead them -
>remember that a programmer is always cheap compared to a manager ). A
.
.
.
Kay, please parse this for me. Are you making the point
that Java managers are intrinsically more available and/or
cheaper than Python managers?
Kay Schluehr
Thomas argues that Python programmers are more expensive than Java
ones. But if one needs more Java programmers to fit into the project
plan one needs probably more managenment/admistration staff ( ~ ratio =
1/3)
and managers are usually more expensive than programmers.
I concede that this is hard to measure but for a similar issue one
should remember that the OPs project did not incorporate the role of a
"software architect" up to now - it did not have to be reified.
Kay
Cameron Laird
No apologies, please. You're making perfect sense,
and I have no argument with what you write here. The
density of pronouns earlier in the thread passed my
threshold, and I simply overflowed. Thanks for the
clarification.