Javascript is my favorite language, but building very large systems composed of interdependent modules is still very tricky and everybody does it differently (especially when you leave the browser and talk about the various server-side JS offerings).
The problem is that various ECMAScript standards committees never really considered "programming in the large". The CommonJS project is important because it aims to fill this gap and create a real JS ecosystem by:
1) Creating a standard for interoperable modules that handles scoping (i.e. module X can't mess with module Y) AND dependency resolution.
2) Attempting to define a set of standard APIs for binary data, files/sockets [streams], and anything else that you might want in a good standard library.
I encourage everyone who writes a lot of JS to join the CommonJS discussion group. Currently, the module API spec is mostly complete (and there are a good number of working implementations). The hot topic now is the Binary data API, which is extremely important because it will be used as a base for all subsequent modules that want to manipulate binary data.
You bring up some good points. The next ECMAScript standard seems to be aiming for some programming-in-the-large goals (though the new features may not be enough to satisfy everyone).
A couple months ago I saw some discussion about CommonJS on es-discuss which signaled that there is actually some influence flowing from CommonJS towards the new ECMAScript standard, with Brendan Eich briefly mentioning some of CommonJS' features. The ES committee is watching and learning from what is going on in the wild, and there is some overlapping consensus now on how JS can improve with regards to programming-in-the-large. This is a very good thing.
I'm curious how well and how quickly all these new server-side JS systems (nodejs, commonJS, V8, etc) will react to the upcoming new ES5. It now seems likely that JS coders will get to try out ES5 a lot sooner on the server-side than on the client-side, given how long it will take (forever-ish) for ES5 to be adopted in browsers.
CommonJS is tracking ES5. For the time being, we vaguely suggest that all compliant implementations must support all features of ES5 that are implementable or stubbable with existing JavaScript features. For example, Object.freeze can be implemented as a stub no-op so people are free to use the API in their modules, and get the feature when it's supported. The intent is to eventually make ES5 compliance a requirement.
Why is Javascript your favorite language? What combination of properties does it have that make it special, apart from the ability to run in the browser?
I mean, it's hardly a bad language, but only the browser thing jumps out at me that says "Use this language for this problem!"
In short:
* It has C-like syntax with cool features like prototypical inheritance, an excellent object-literal format, first-class functions, and closures.
* Everyone knows it (or thinks they do).
* There are some really smart people in multiple different camps working on making extremely fast interpreters.
Still skeptical about JS as a general purpose language? Watch Doug Crockford's "Javascript: The Good Parts". If he can't convince you, no one can.
The biggest problem with "Javascript on the server" right now is that Javascript has no concurrency story whatsoever. You could easily hack in shared-state threading, but there would be no faster way to turn me off of Javascript forever than to do that. You could hack in message-passing, but it won't be clean and you're still going to have terrible fun trying to not pass mutable references, which, again, if you leave that in I will run screaming. STM isn't exactly going to be easy to stick in either.
(I say I will "run screaming" not merely because I don't want to deal with it, but because I don't want to deal with yet more libraries based on shared-state concurrency. No. I've left that behind and I'm not going back, and certainly not in a space where I've got an abundance of languages to choose from.)
Without a solution to that problem, I see this as a terribly, terribly wrong move, and a terrible time for JS to try to be picking up the server. Servers are in the multicore age and have been for a couple years now.
(And if your answer is "well, we'll just keep all the tasks simple and we won't need multiple threads", I would remind you that you're running on the server, which often needs to do, you know, actual work. A language that can't do that may have a small niche, but it's going to be an uphill battle against entrenched competitors.)
One solution that will almost certainly get implemented is WebWorkers. From that you get string-only message passing (though it'd obviously be easy to do json-encoding of data in messages) between js processes.
That, combined with event based non-blocking apis for managing multiple tasks neatly covers more or less everything you can do in many other mainstream languages like Python and Ruby (excluding some magic in native code extensions).
That makes some sense. I would agree that it should be optimized to specify that JSON-able objects should be passable in such a way that it is if they are being fully serialized out to strings then deserialized again, giving the implementation the ability to do some optimization (which may or may not be a good idea but you'd have to try it to find out, best not to write yourself into a corner), but actually doing that in a library would be easy enough.
Given the way that JS performance has been moving, that would indeed be able to take on quite a workload, if you can run a decent number of "webworkers" at a time.
What use cases are you envisioning where threading will be essential? Or aiming in the opposite direction, what use cases are people envisioning for server-side Javascript at all?
I presume its anticipated niche is as a CGI-type replacement --- relatively light-weight 'programs' executed per request. If that's the case, is there much need for multithreading?
"What use cases are you envisioning where threading will be essential?"
In 2009, my default presumption is that an emerging platform that probably won't even be viable until 2010 ought to have a concurrency story better than what we already had. If it doesn't have that, why wouldn't I use an existing language?
If it were entering a vacuum, sure, great, no problem, but it's not.
I guess this goes back to my old standard for examining the usefulness of a language: "What problem does it solve?" There ought to be a single sentence that says what problem the language solves. Python: "It's too hard to program; we need to make it easier." Perl: "Too many separate UNIX tools, we should merge them together." Haskell: "Our academic theory is great and all, but we need to come out of our ivory towers and prove it works.
Javascript on the server: "..."? "I really want to be able to use the same language on the server as the client" is a I think what it's going for, and as far as that goes it's ok, but my objection is that it isn't a very good server language. And note my objections really are about the language, not merely a lack of libraries, which is completely solvable. You can't just graft on concurrency anymore, at least not as far as I'm concerned; my standards have risen.
While I agree with you that the lack of concurrency is problematic in the long run I think that JavaScript on the server will be at least as useful for many web applications as PHP, Python, Ruby, Perl - all of which have modest or non-existant concurrency stories.
I'm curious as to why you think JavaScript can't hold it's own weight against said-languages for the server-side?
there was no consensus and the topic was delayed for later discussion - but it seemed the promises idea got most attention (my pov) http://wiki.commonjs.org/wiki/Promises
Arguably you can use webworkers (available on V8 and JSC, and thus Narwhal on those platforms) for concurrency problems. Webworkers enforce no shared state though, they use a message passing mechanism. Not the greatest solution.
Concurrency is important, but I think most server tasks don't require it. JavaScript doesn't need to be the 100% language to "win" it just needs to be the 80% language.
What I want is an extension for my web framework that transparently pre-renders certain Javascript-generated parts of the page, and then sends the browser a serialized version of that page that includes not only DOM structure but fully wired-up event handlers.
This should be possible using HTML and <script> tags, but will take some serious work to become plausibly cross-browser. But if that work happens, there will be a way to automatically provide the usability/speed benefits that previously required dual client & server implementations without the attendant cost. And that will drive a lot of people to using whatever standards those tools are built on.
This is kind of cool, but who is going to digest all of it? The techno folks may have some technology pieces working, but nobody out in the main world would be able to figure it out. I got a little dizzy just looking at all the links to all the pieces in just this one article. Let alone actually reading them all.
I like Java script but I think making it go server side is backwards. Why not make Python or Perl or Ruby go client side instead? (I am sure plenty of folks smarter than me have already thought about and probably worked on it.) Those languages already have all the parts they need except a connection to the browser dom. Java script has a connection to the dom and practically nothing else. These other languages have been getting tweaked for years on the server side. It just seems like such a big byte to get Java script to work as a regular language like the others. It also ignores some of the things that are lurking like Erlang and Clojure. It sounds correct that there are lots of Java script programmers. However are there lots of Java script only programmers? I doubt it. While it is compelling to write one language every where, I dream about it sometimes, we aren't close yet.
So embedding the Perl, or whatever, interpreter into the browser is probably a chore, but it is also probably doable, the browser should be able to deal with multiple languages. I think it is just that the people that are capable of doing that job looked at it and said. "Crap! I will just learn Java script." So the reality in my mind is that programming in two or three or four languages is just easier than doing all that other work. Everybody does it. In fact we say I know Java, or Python, or Java script, but that is just a word to give us a frame of reference when we communicate with each other. In fact what we really know is how to program computers and Python is a subset of the total global programming skill that I have. It is a like speaking English in some specific domain. There is jargon everywhere, one can speak English but one tends to get easily lost if one listens to lawyers speak to each other or doctors, and one is also not a doctor or a lawyer. Does that mean we should create a generalized English that both doctors and lawyers can use in each of their specific domains? Seems a little silly when one thinks about it. Is making Java script work everywhere is really buying all that much?
One quick note: its "JavaScript" one word, no spaces. Java and JavaScript are two very different languages putting a space there would only infinitely confuse everyone even more.
JavaScript is a beautiful language with much more to it than just the DOM API. You seem to suggest that this is JavaScript's only strength. In fact it could be argued that the DOM API is the single greatest problem with JavaScript in the browser[1].
It's true that different languages have different problem domains for which they are well suited. I would say that projects such as Node.js[2] prove that JavaScript has applications outside of the browser.
Going the other direction (moving server-side languages into the browser) is simply not feasible with the current rate of browser development and adoption. It would be at least 5 years with a concentrated effort to accomplish such a task.
I agree that the effort is large, and the browser platforms are constantly in motion. I was mostly just musing about it. My gut feeling is that getting Java script up to speed on the server side is just as much if not more work. Look at how much work has gone into the tools just for Java for example. Years. Before I get flamed, I will say that I am not bothered at all that some folks are willing to take up the effort to do it anyway. That is how things get done. I am just wondering about it.
MS has made huge strides toward this w/Silverlight. You can run C# directly in the browser, and last I looked (admittedly some time ago), the performance trumped JS in the browser by several orders of magnitude.
I'm glad the JSGI spec is fleshing out, the web connector API will allow many more frameworks to get started easily, rather than forcing Javascript programers to hack on java or C to build a webserver first.
Once JSGI is done, it really will allow the experimentation you see from Rack and WSGI in the respective Ruby and Python worlds.
The CommonJS module system is really a step forward.
For the last two months I've been developing a webapp on Helma NG and I like it very much. Parts of the framework fit together nicely. The community on IRC is very helpful and provides answers in the rare cases where source code and documentations is not enough.
I also want to praise the project developers for their code quality -- even though I use a version from git I haven't had a single problem with frequent updating and re-building it.
The problem is that various ECMAScript standards committees never really considered "programming in the large". The CommonJS project is important because it aims to fill this gap and create a real JS ecosystem by:
1) Creating a standard for interoperable modules that handles scoping (i.e. module X can't mess with module Y) AND dependency resolution. 2) Attempting to define a set of standard APIs for binary data, files/sockets [streams], and anything else that you might want in a good standard library.
I encourage everyone who writes a lot of JS to join the CommonJS discussion group. Currently, the module API spec is mostly complete (and there are a good number of working implementations). The hot topic now is the Binary data API, which is extremely important because it will be used as a base for all subsequent modules that want to manipulate binary data.