Contents
- I Ranted and Eric Rebutted
- While I Respect the People at Microsoft…
- …They Become Detached
- Eric Unconsciously Supports my Thesis!
- Dismissing the Proposal, Not Solving the Problem
- Not More Power; Transitionality!
- Today’s Potential Didn’t Address Yesterday’s Deficiency
- Nor Does an Orphan Address Yesterday’s Deficiency
- And a Potential isn’t a Solution
- Yes, there’s Powershell, but…
- A Correct yet Irrelevant Point
- Interest Doesn’t Necessarily Change Process
- A Solution Offered; Wrong Product, Years from Now
- Too Little, Too Late: Acknowledged
- Studying Users Isn’t Feeling their Pain
- Trade-offs that Shortchange Developers
- Experiencing Pain Empowers Real Solutions
- Yes it is a Paradox…
- …But Solve it You Must, Or Else
- Change, or Be Changed
- What if I’m Not Wrong?
- Comments
I Ranted and Eric Rebutted
The day before yesterday I wrote a long winded and rambling rant about how Microsoft’s release cycle and process for creating developer tools. I commented on how I believe it is making them fall behind and causing many formerly loyal Microsoft developers to look at open source solutions on non-Windows platforms. I referenced a post that Microsoft’s Eric Lippert wrote over 2.5 years ago titled Top Minds Are Working On It[1]. In retrospect it might have appeared I was being critical of Eric but that wasn’t my intention, and if that’s how I came across I apologize. Instead I was referencing Eric’s comments as symptomatic of the Microsoft culture at large. And yesterday I awoke to find that Eric had issued a rebuttal.
While I Respect the People at Microsoft…
But before I address his comments let me talk about Eric and all the others I’ve met from Microsoft. Eric is actually a brilliant guy, and very likeable. I’ve met Eric face-to-face and it’s obvious he’s much smarter than me. But then I could say that about most of those I’ve met from Microsoft; they don’t hire dummies. As a rule I’ve been impressed with every Microsoft employee I’ve met. They are super bright and AFAICT they do really want to do “Good Things(tm)“.
…They Become Detached
But group dynamics being what they are, when you get a group of super-bright people together they become competitive, hone their debate skills, and learn to be strong advocates for whatever their own positions. And they can become detached from the outside world, much like politicians in high office. Politicians are also typically super bright, and most enter office wanting to do Good Things(tm), but once “on the inside” they loose touch with the concerns of their constituents. So I am not condemning Eric and his Microsoft colleagues, I am merely commenting on the culture that results and collectively channels them.
Eric Unconsciously Supports my Thesis!
So I started yesterday’s post saying I’d been planning to blog on the topic for a while but the reality was I still didn’t know how best to explain my concerns. But for better or for worse I did write a rambling essay yesterday, but ironically Eric’s comments made my points far better than I! My central thesis was that Microsoft isn’t meeting developer’s needs because of their processes and infrequent releases and consequently open-source alternatives are meeting developer’s needs instead. Eric’s both debated my examples and pointed out they now plan to address some issues I referenced. But not only did Eric not address my central thesis, he ironically supported it given his rebuttal’s choice of focus! In my post I wrote the following:
“Microsoft’s culture is to argue semantics when reality doesn’t match their world view”
And Eric’s comments proceeded to do exactly that! In my post from 2.5 years ago[1] I called for Microsoft to address things that PHP, Ruby, and Python are addressing today, which Eric rebutted at the time. In yesterday’s rebuttal Eric referenced his earlier comments stating (emphasis mine):
“Second, my ‘esoteric’ reasons for not implementing a scripty version of VB on the .NET platform were hardly esoteric then and are hardly esoteric now. They are (1) fracturing the language further causes confusion amongst customers and is massively expensive for little gain, …”
Dismissing the Proposal, Not Solving the Problem
Now I’ll freely admit Eric is far more qualified to evaluate my suggestion on technical merit, but that wasn’t the point of my 2.5 year old post. Customers with needs a company’s not addressing will often propose solutions they believe will address their needs, yet often their suggestions aren’t workable for whatever reason. People who specialize in addressing customer needs know that rather than dismiss suggestions as unworkable it’s far better to determine the customer’s actual needs and implement a workable solution instead. And often, many other customers have those same needs. So Eric dismissed my proposed solution but didn’t address my unresolved needs that prompted the proposal. I don’t attribute this failing to Eric, I attribute it to Microsoft’s current culture.
Not More Power; Transitionality!
Eric then went on to say:
“(2) we can do things to make VB.NET more powerful without fracturing the language,…”
Ironically, I didn’t ask for a more powerful VB.NET; I asked for one that was easier to start using and one that developers could then easily transition to more powerful usage. Though they believed they were providing an easier to use Visual Basic 2005, they addressed the language but not how people develop applications. Though they made strides with the Express Edition, my biggest concern was with the complexity of the development environment and the language. I suggested an interpretive environment with a transitional language design that allowed new developers to start easily yet be able to effortlessly grow their expertise with use. What I envisioned was something like Boo, but I wanted it 2.5 years ago with a simple interpretive environment, and I wanted it from Microsoft so that it could possibly generate a large and immediate and user base with a thriving community and significant peer support.
Today’s Potential Didn’t Address Yesterday’s Deficiency
Eric continued with the following…:
“(3) Microsoft makes scripting languages like Iron Python”
…but omitted the fact that a robust Iron Python was just a gleam in Jim Hugunin’s eye 2.5 years ago and is still not ready for prime time. Further, Microsoft’s approach is to host IronPython in Visual Studio which does nothing to bypass the complexity of Visual Studio!
Nor Does an Orphan Address Yesterday’s Deficiency
Eric then said:
“…and JScript .NET, use the right tool for the right job.”
To which I did a double take wondering if he were really serious! JScript .NET is such the orphan that I can’t even believe he suggested it! The JScript .NET newsgroup has less than a screen full of messages, JScript .NET hasn’t been updated since 2003, and nobody’s even written about JScript .NET in years! So could Eric really have been serious when he suggested JScript .NET? Well, assuming he was, then:
- JScript .NET does NOT have an easy-to-use interpretive environment, and
- JScript .NET is a complex language; NOT simple-to-use, and
- Again, JScript .NET has NO user-base!
And a Potential isn’t a Solution
Moving on Eric said:
“The .NET framework is already amenable to the development of scripting languages.”
So why do we still not have a viable scripting solution for .NET supported from Microsoft more than half a decade after the .NET Framework’s first release?
Yes, there’s Powershell, but…
Okay, that’s not quite true. Eric didn’t mention it but in the spirit of honest debate, there is Microsoft’s PowerShell. But while I will freely admit PowerShell is really nice, PowerShell:
- Was only just released so doesn’t address the past 2.5 years,
- Doesn’t have a development environment,
- Can’t be used for web development,
- Doesn’t have a compiler for creating components for use in other .NET languages.
- Doesn’t have transitionality allowing it to scale up for much more complex projects as the developer’s experience grows.
A Correct yet Irrelevant Point
Eric then makes a point about “scripting languages” vs. “dynamic languages” (emphasis mine):
“Third, I want to make a distinction between scripting languages (languages intended to script things) and dynamic languages (languages which admit a type system which cannot be deeply analyzed at compile time.) Scripting languages are often dynamic languages, however it is entirely possible to use dynamic languages for tasks other than scripting. “
Okay… So Eric’s points are very technically valid, but they are totally irrelevant! Frankly I wasn’t asking for a language that was “intended to script things,” I was proposing a language (and IDE) that would be:
- Productive,
- Easy to start using, and
- Scalable as one’s skills evolve.
Call it “scripting“, call it a “dynamic language“, call it whatever; it’s irrelevant. What is relevant is for it to be productive, easy, and scalable. Microsoft could choose to get there however they will, bit like arguing the semantics of “Car” vs. “SUV” with someone who just needs transportation, Eric’s distinctions were simply irrelevant to the needs. Totally unrelated, I ran across this joke yesterday. What could be more ironic?
Interest Doesn’t Necessarily Change Process
Eric finishes his prior point with:
“The VS team is VERY interested in understanding how to make the platform more amenable to dynamic languages.”
Great! But are they going to actually engage people who are not .NET developers in the design of said dynamic languages and their respective development environments and then incorporate their feedback? Or is the VS Team just going to plug another dynamic language into Visual Studio? If the latter they will do so ignoring that Visual Studio users already have the language(s) they need and that there are at least an order of magnitude more people for whom Visual Studio is too overwhelming.
A Solution Offered; Wrong Product, Years from Now
A couple other comments Eric made were:
“C# 3.0 will have the “one line auto-implemented properties” feature you requested for VB. Enough people asked for it, we put it in. I do not know if VB will be doing the same. You’re welcome. “
and
“Current C# 3.0 features move in the direction of dynamic languages without actually making the language dynamic (lambdas, improved generic method type inference, extension methods, implicitly typed locals, anonymous tuple types). All of these however are implemented so as to keep the language statically analyzable. We are considering features for C# 4.0 which would make the language more dynamic without losing that important statically analyzable core.”
That’s well and good, but it doesn’t address VB.NET, and it also makes my own point that Microsoft’s release cycles are too far apart! There are badly needed enhancements and they need to get them to developers more often than once every three years. C# 3.0 is still a way’s out, people need something today, and I personally wonder if I’ll even care about programming by the time C# 4.0 is released. Hell, I hope to have made my fortune and be retired by then! :-) But seriously, C# is a professional developer’s language and adding features to a professional developer’s language wasn’t even close to what I was proposing.
Too Little, Too Late: Acknowledged
In the last paragraph, Eric finally gives tacit acknowledgement of the concerns I raised in yesterdays post:
“Now, maybe these features aren’t what you want, or are too little too late, or the release schedule is too long for your liking, or whatever. That’s unfortunate.”
Exactly. The release cycle needs to be compressed by an order of magnitude as it has been at the competition. More on this in a bit.
Studying Users Isn’t Feeling their Pain
Eric then signs off with:
“However I take exception to the claim that we do not study what real users are doing and try to provide tools that do what they want. We do that every day.”
With this Eric either misunderstood my point, or more likely I didn’t state my point in a manner that was understandable. Whichever the case, let me clarify. I know the VS Team works hard to study real user’s needs every day. But what I also know is that the people who make the decisions about what gets released and when have considerations that are far different from the needs of developers. And it is human nature for one to place solving one’s own pains ahead of solving the pains of others as people simple can’t fully comprehend pains they don’t experience.
Trade-offs that Shortchange Developers
Eric let’s be concrete; if solving a customer problem today will cause the VS team a problem tomorrow they won’t do it. For example if the VS Team plans to rearchitect something next version they won’t provide an interface that developers need in this version if doing so will make that rearchitecture difficult. Some would say this is good product engineering to which I would actually agree, but it is nonetheless a trade-off that keeps developers from getting what they need today.
Experiencing Pain Empowers Real Solutions
So, if you’ll reread my post from yesterday you’ll see that I didn’t say “Microsoft doesn’t listen to customers“; I know damn well they do. On the contrary, I said that Microsoft’s Developer Division “Don’t solve real world problems.” And the reason you don’t is because you don’t experience the pain that those real world problems cause. By comparison most developers contributing to open-source projects are doing so to solve pains which they themselves have, and that is why they are addressing developer needs much faster than Microsoft’s Developer Division.
Yes it is a Paradox…
An observant person would say that I’ve present a rather intractible dichotomy for Microsoft’s Developer Division; i.e. they need to use the developer tools they build to solve real world problems yet they also need to to develop those tools ten times faster! Now I’m sure these comment will cause the blood of those in Microsoft’s Developer Division to boil thinking I believe it’s possible they do their work ten times faster and do real world projects. But I was not advocating that; I’m fully aware of the myth of the “man-month” in software development projects.
…But Solve it You Must, Or Else
What I was pointing out, however, is if Microsoft’s Developer Division maintains its status quo they will slip farther behind and the loss of developers to other platforms will accelerate. And since developers, developers, developers have always been Microsoft’s life blood it is critical they address this issue. Every developer they loose to Linux or the Mac doubly weakens Windows. Microsoft must address this issue if they are to maintain the same level of relevancy during the next twenty years that they’ve had for the past twenty years. And most of what I’m suggesting isn’t complex, and it isn’t new. They probably have most of what they need already written and used internally. They just need to rethink what they are offering.
Change, or Be Changed
So to wrap up this second long-winded essay:
Microsoft’s Developer Division needs to implement drastic changes sooner than later. If they do not, outside forces will soon impose drastic changes upon them and it’s certain they will find those imposed changes to be far more painful. Or as the mechanic on the old Fram Oil Filter commercial used to say “Pay me now, or pay me later.”
What if I’m Not Wrong?
P.S. If you’re from Microsoft’s Developer Division and choose to dismiss my concerns, just ask yourself this: What’s the downside for you if I’m right?
- Eric Lippert’s post was in response to my post entitled A Modest Proposal: VBScript.NET, aka “Helping Mort use .NET”.
Hey Mike,
Well if your central points are in fact that VS releases are too far apart and that the processes we used during the "Whidbey" release led to the release taking 36 months when it probably should have taken 18, I think you will find not a single person of the thousands who work on Visual Studio who will disagree with you. I lived through those two solid years of bug fixing and it was painful every day, and when we were done with it everyone knew that we had to fix it or we would never ship a quality product on time again.
So we did. We took a step back and spent several months doing two things. First, the developers and testers spent those months doing all of the quality infrastructure work that always gets postponed because it doesn’t directly affect the product that is immediately about to ship. Second, the program managers and management studied our processes and figured out how we got into the position of creeping features and integration nightmares that caused Whidbey to take so long.
We are trying lots of experiments in Whidbey with our processes to see what techniques address these issues, and I think we are doing a reasonable job of it. We’re keeping the postponed bug debt low, the quality high, we’re massively increasing the amount of developer testing that gets done, we’re borrowing a lot of ideas from agile methods like scrum, blah blah blah, I could go on all day about quality gates and feature crews and our new integration model and where it works and where it doesn’t, but I won’t bore you further.
My point is that we’re not idiots. We live with the pain suboptimal processes and long release cycles every single day, you just see the result. We know better than anyone else what is broken. You’re right — we needed some fairly drastic changes to our processes. So we put them in place _over a year ago_ now. If Orcas ships on time then we’ll know whether they were the right changes or not.
Eric (how do you get notified of my posts so fast?):
Rather than make another long rambling post, please realize that I made this and the prior post as background for some suggestions I plan to make that are nothing like what you are currently doing (and that might not require you to change what you are currently doing that much, if you can believe that.) But I didn’t want to announce the post because whenever I do that I never seem to get around to making the post. Anyway, I plan to write a post with numerous "out-of-the-box" suggestions. Hope I can write them soon.
BTW, the more I learn about web technologies, the more I think you guys really took the wrong path with your web technologies including with VBScript, more so with IIS, and especially with ASP.NET. I plan at least one several posts on that topic.
Couple more short takes.
* When I said "we are trying in Whidbey" of course I meant Orcas.
* JScript.NET now has a team that owns it again. The uptick of interest in the JScript language thanks to AJAX has not gone unnoticed. I do not know of any specific plans to enhance JScript.NET support, but the fact that someone actually owns the code again is a heartening step.
* VS Whidbey was explicitly designed around the transitional model you suggest. That is what features like "just my code" address; they massively simplify the experience for novice developers by hiding features and information which the novice developer is unlikely to either understand or care about. The granularity at which things can be ratchetted back on is in my opinion if anything TOO granular. But certainly the criticism that VS looks like a 747 cockpit when it should look like an SUV dashboard was heard years ago and we have been working hard to address it.
I agree that it is nice to have languages which support that kind of transitional development as well, and this is one of the principles we designed JScript.NET around. The C# 3.0 LINQ features I feel are quite well designed in this regard; our considerable improvements to intellisense and the expression evaluator (ie, watch window) should help here as well.
I wish C# had a few more syntactic sugars to make writing small, scripty programs easier, but syntactic sugars are _very_ expensive so we want to make sure that we get exactly the right ones to get the best bang for the buck.
IronPython 1.0 released earlier this year, so I’m not sure I understand what you mean by "still not done".
IronPython also doesn’t require Visual Studio to be used (contrary to your statement above). You can optionally use the Visual Studio web project support for it if you want, but this is optional and not required.
Eric, thanks for the additional comments (man you write fast):
Something I think you’ve overlooked on my points is that you need another development tool Visual Studio. Call it the "Power Series" if you will (given PowerShell), but something that meets the needs of "Occupational Developers."
That category is not addressed by you now, and I’ll bet it’s potentially much larger than professional developers albiet they won’t spend as much, but they will build for Windows. And occupational developers are not hobbyists; hobbyists do it for fun and learning, occupational developers do it to accomplish something. Although I have past experience as a professional developer, I am really an occupational developer.
It probably needs to be another team working on it, but you should share the same languages, and the transitionality requirements of the "Power Series" team should have as much clout with the language teams as the "Visual Studio" people.
BTW, this is about 10% of what I had planned to suggest. I’m also planing to write a "Microsoft: The 85% solution" because you take us 85% of the way and then leave us with either a brick wall or a huge climb when it would not be that hard to take us 100%. Looking at Apache/PHP, I see none of that; they solved all the little problems I had in IIS/ASP/VBScript, but that’s also the impetus for some of my recommendations.
Lastly, can you explain why syntactic sugar is so expensive, and contrast how that expense is paid by open-source solutions that add-in that syntactic suger? BTW, I read this apropos comments on a blog post recently: "They say that in programming languages, common design patterns make up for missing parts of the language." (http://www.webstandards.org/2006/11/07/have-your-say-about-the-future-of-html/#comment-11244)
Hi Scott:
The reason I said what I did about IronPython was the screen cast I saw on Dr. Dobbs at http://www.drdobbs.com/dept/windows/194500116?cid=RSSfeed_DDJ_Windows/.NET where Scott Swaggart said that "there is no intellisense for the .NET Framework", "the compiler errors were not that useful", and that "it is an interesting novelty but I don’t think it is ready for prime time development." (I should have referenced that video.)
As for programming for Iron Python without w/o Visual Studio that means having no development environment, no debugger, etc. One of my biggest complaints is that Visual Studio is too complex but an alternate development environment is needed.
Sugars have many costs.
Consider, for example, the entirely unnecessary prefix and postfix increment and decrement operators in C. They are clearly sugars. They are bad sugars. Their costs massively outweigh their benefits.
First off, C didn’t even specify them well enough to make them portable across language implementations. There is no telling what M(++a, ++a) does because the language specification says that it’s legal and the compiler can choose to do the evaluations in any order.
Now, one might say that this is just an unfortunate oversight, and it is, but that would miss the point. The expense of the sugar at language design time is what Wesner Moise calls "n squared development". Adding the nth sugar means that the interactions with the previous n-1 sugars have to be analyzed and implemented correctly. This is true of all features, not just sugars, but sugars are particularly heinous.
But now consider how that choice of sugar affected future development of the C++ and C# languages. ++ is an operator. That means that when operator overloading was added to C++/C#, you kinda had to provide a mechanism to overload the ++ operator. It would have been weird not to. And now suddenly we have all the problems associated with operator overloading; we have no guarantee that x=(++a), x=(a+=1) and x=(a=a+1) assign the same value to x anymore! This sugar strongly urges the language designers of future languages to make choices which they would not otherwise have made.
This sugar also caused pain when we designed and implemented expression trees in C# 3.0. The expression tree code I wrote is more complex, and hence more likely to be wrong, than it would be if it didn’t have to deal with all the ways in which a ++ can be illegally hiding inside an expression tree.
Now think about how ++ interacts with automatic semicolon insertion in JScript. The existence of postfix operators in a language with automatic semicolon insertion leads to some crazy ambiguous situations and massively complicated the JScript grammar as a result.
This is just one example of a particularly bad sugar. The thing is, the guys who designed C had no idea whatsoever of the pain they were causing legions of programmers and language designers in the future. They just figured it would save them a few keystrokes here and there.
We think VERY VERY HARD before we add a sugar. Sugars can have massive unintended effects that percolate down through the ages across multiple languages. This is why we thought about one-liner autoimplemented properties in C# for a couple years before deciding to do it. We have a duty to our future selves to not screw up the future development of the language or any descendant languages.
Eric: thanks for the details about sugar.
But I want you to think about something for me if you will. All your examples related to the concerns of the language designers and implementors. How many language designers and implementors are there in the world who are actually getting paid to do that? Now how many developers are there out there? 4 or 5 orders of magnitude more. So when you delay adding something significantly valuable to a language, you are negatively affecting the productivity of millions of people to save yourself from difficulty later. You are doing exactly what I said above; you are making a trade-offs that reduces your pain but that that shortchange developers.
No don’t think that I don’t appreciate this from all angles. If I could be I’d love to be a language designer; for some reason it just fascinates me. But I don’t have anyone paying me to do that, and I don’t want to work on a language that only a few would use. So I can see your concerns. But on the flip side I can also see the fact that by taking so long to come out with new things, you are affecting many orders of magnitude more developers in a negative way.
I’ve been thinking about this issue a tremendous amount over the past six months and come to realize that there are so many cases where you give us almost what we need but not quite enough. A perfect example is the inability on IIS/ASP.NET to do URL rewriting on a URL that does not have a file extension. It would have been so easy (in the grand scheme of things) to provide that but you guys didn’t and many man years has been invested to try to get around that omission, yet it’s a non-issue on Apache. (I could go on, I have about 10 more of these…)
Also, a point on your example. You gave an example that affects expression syntax and I would agree those are very challenging. OTOH, in VB.NET at least, statement syntaxes have no where near the potential ripple affect. My kingdom for a one line property declaration in VB.NET!!!
Anyway, I could "complain" all day about the fact you’re not delivering fast enough, and you can "justify" why you are not all day long but the reality is that the language and frameworks in the open-source world are advancing at a rapid pace and the worst thing for you guys is for guys like me to have quit complaining. If we do, it’s because we’ve given up on you and switch to Linux and PHP. I can tell you I’ve switched some of my projects to those platforms, and it costs me more in hosting.
But I simply can’t accomplish my goals with what you are delivering. And I don’t expect you to care about me, but you should care about what I represent; lots of disenchanted ex-Microsoft loyalists.
Which is why I hope to be able to write my series of suggestions in the near future (I have to percolate on them a bit more before I do.) You guys may ignore them, but at least I’ll be able to say I tried. (Or maybe you could hire me to come and implement them. ;)
One last question: How do you see my comments so fast? Do you have a feed reader that reads comments? And were you still subscribed to my blog after not having crossed paths for a few years?
Your comment here doesn’t really make any sense:
"there is no intellisense for the .NET Framework"
Scripting language (including PHP, Python and Ruby) typically don’t have intellisense for APIs, since in a scripting world you often can’t infer the type. Instead they usually have color coding of types and keywords (which IronPython has) and sometimes syntax completion.
IronPython works in the free Visual Web Developer download, and includes built-in WYSIWYG designer support and debugger support (something PHP, Python and Ruby both have very weak support for). I think you’ll find that although still new, IronPython actually has much better tool support than almost all other scripting languages out there. It also allows you to build any type of application using .NET (ASP.NET, WPF, WinForms) which is very powerful.
I take it from your comments above that you haven’t actually ever tried it. You might want to consider playing with it or researching it before commenting on it further.
Note: I am a Python developer, and the *only* interest I have in .NET is IronPython.
Scott wrote:
"… debugger support (something PHP, Python and Ruby both have very weak support for)"
I guess you are kidding? Python has Pdb built-in which I consder the first rate debugger, which is even extensible. Obviously, you never used Winpdb either, which does thread debugging, secure remote debugging, arbitrary namespace modification and whole lot more in addition to Pdb.
IronPython doesn’t support Python debugging infrastructure, so it has *poorer* tool support from the Python developer’s point of view. It doesn’t support Python AST compilation, which makes hosts of code analysis tools useless. It doesn’t support Python profiling hooks, so none of profile, lsprof, statprof work with IronPython. It doesn’t support Python tracing hooks, which makes trace and coverage analysis tools useless. IronPython has too many features missing which Python programmers take for granted, it’s not amusing.
Let’s quote you again:
"IronPython actually has much better tool support than almost all other scripting languages out there. (snip)
I take it from your comments above that you haven’t actually ever tried it. You might want to consider playing with it or researching it before commenting on it further."
Shame on you.
Seo,
The VS debugger supports all of the features you listed above, and allows you to step between multiple languages (Python->C++ or Python->C#->Python) in a single call-stack. The .NET profiling APIs and code coverage tools are also excellent.
Mike’s assertion that .NET doesn’t have scripting language support is just wrong.
Mike wrote:
"Lastly, can you explain why syntactic sugar is so expensive, and contrast how that expense is paid by open-source solutions that add-in that syntactic suger?"
Syntactic sugars are expensive. Let’s try a simple example. How does your "Into" property syntax work with XML documentation in comments and tools to extract them? Surely, you want to document properties?
Let’s try an open source example. Python has been long criticized for lacking a conditional expression. You know, as in "bool ? then : else". That should be trivial, right?
It required months of debate, at least 10 different syntax proposals, and when implemented, it required an obscure hack in the grammar to be backward compatible. Read PEP 308 for full details: http://www.python.org/dev/peps/pep-0308/
Summary: conditional expression, "A if B else C", clashes with generator expression, "A for B in C if D", where C is unparenthesized lambda sequence because the precedence is "conditional > lambda > generator". I bet *nobody* thought of this up-front.
See also PEP 342 for a (very good) syntactic sugar introduced to Python, to allow "yield return" to be used inside expressions. It shows what kinds of design issues are involved in introducing syntactic sugars to programming languages, not only open source ones. Mind you, PEP 342 needed two wholesale revisions to reach the current status: http://www.python.org/dev/peps/pep-0342/
Scott wrote:
"The VS debugger supports all of the features you listed above, and allows you to step between multiple languages (Python->C++ or Python->C#->Python) in a single call-stack. The .NET profiling APIs and code coverage tools are also excellent."
That’s rather useless to me, since that implies I need to learn new set of tools to debug, profile, trace, cover, lint, extract strings for translation, generate API doc and online help, in .NET way, while Python already has mature set of tools to do all of them and I know how to use them well.
The correct solution is to support underlying hooks to do above in IronPython. It *is* part of Python-the-language.
I admit debugging multiple languages is a nifty feature, but when you develop in Python, the typical scenario is to write >95% codes in Python and the rest in C only if necessary, so it’s pretty marginal. I think that’s what Eric meant by "dynamic language vs. scripting language". Almost all uses of Python is as a dynamic language, not a scripting language.
"Mike’s assertion that .NET doesn’t have scripting language support is just wrong."
Oh sure. IronPython is a very good and mature *scripting language* for .NET. It is even a good *dynamic language* for .NET, which is productive, easy, and scales as your skill grows. Mike, please try it, really.
It is just *not*, empathically not, a mature implementation of Python expected by Python programmers. In addition to above missing pieces, it’s not compatible in some cases, so it doesn’t run ZODB, doesn’t run Twisted, doesn’t run SQLObject, which are bread-and-butter libraries of Python programmers. But I guess nobody asked for that, oh well.
I’m part of a development team for Resolver Systems writing a large desktop application almost entirely in IronPython.
IronPython is an excellent language implementation, which the Microsoft team have used (amongst many other things) to illustrate and improve the Visual Studio support for alternative languages.
Although Visual Studio has pretty much full support for IronPython, we don’t use it except when we want the debugger (which is rare).
I disagree with the Dobbs journal article which says that IronPython is not ready for the prime time, and I think that to assert that as fact without even trying the language is ignorant. :-)
Other IDEs (like Wing) provide autocomplete for Python and can be used with IronPython.
We’ve found developing with IronPython an excellent experience, and are impressed with the Microsoft development team. At last it *does* bring the flexibility and development speed of dynamic languages to the .NET platform, which seems to be at least partly what you are asking for.
Barrier to entry is low, IronPython is suitable for simple scripting tasks, and it does scale to large applications.
Seo
""The VS debugger supports all of the features you listed above, and allows you to step between multiple languages (Python->C++ or Python->C#->Python) in a single call-stack. The .NET profiling APIs and code coverage tools are also excellent.""
"That’s rather useless to me, since that implies I need to learn new set of tools to debug, profile, trace, cover, lint, extract strings for translation, generate API doc and online help, in .NET way, while Python already has mature set of tools to do all of them and I know how to use them well."
Well, PyLint run from CPython can analyse IronPython code with no problem.
Have you tried using the VS debugger Seo ? It’s not very difficult… :-)
Seo Sanghyeon wrote:
>> Syntactic sugars are expensive….
Thanks for the explaination and links.
That said, please understand my view of expense. I see the "expensive" an expense of language provider (Microsoft for VB.NET and the community of Python implementors for Python) NOT of the language user. And for a language that is used by a large number of people, the "expense" implementing the sugars are far outweighed by the expense to *not* implementing syntactic sugar when you look at productivity losses for the broader developer community including the bugs they need to fix for lack of the sugars that could otherwise have avoided. My concern is for the expense born by the broader development community, not the expense for the language provider.
Please understand it’s not that I’m unconcerned about the plight of the language developer. Instead I’m simply choosing to place a higher priority on the plight of the development community at large because of the greater benefits that would follow.
What’s more, my main thesis was that either Microsoft provide these sugars or someone else will. And I am seeing a lot more interest in open-source languages and web frameworks than ever before because those languages and web frameworks are meeting people’s needs, not scolding them for the expense that addressing those needs would cost the language provider.
Ironically you state that it took "months of debate and at least 10 different syntax proposals" to implement Python’s conditional expression. As if! You are so making my point!
Regarding features in VB.NET, one can only wait years and hope that Microsoft sees it as important. I proposed a simplified property syntax over 2.5 years ago and at this point I don’t believe that are currently considering it. Even if there were, it would probably be several more years before I could use it in a shipping version of VB.NET, but likely it is at least half a decade away if not more, assuming it ever happens.
If VB.NET where an open-source language and it were important enough to me, I could rally support for it in the community and then do the development to add it to the language. It might take months, but I’d still be able to use it in my lifetime. :)
Oh, Seo, can you please include reference to your blog when you comment?
Scott:
>> I take it from your comments above that you
>> haven’t actually ever tried it. You might
>> want to consider playing with it or researching
>> it before commenting on it further.
I love the irony of your criticism. I take it from your comments you really didn’t read my posts nor my reply? The issue of IronPython was seriously tangential to the thesis of my post and I was only referencing it because Eric Lippert brought it up. The issue at hand was Microsoft’s Developer Division’s processes and release cycles. I will admit I allowed the topic to stray and I also discussed how Microsoft Visual Studio is too hard for the "occupational programmer", the latter of which is the more relevant of the two related to your comments. I separated those issues, and I’ve since discussed the "occupational programmer" issue at length[1]:
But to address your point, YES I did try IronPython. I downloaded the IronPython IDE[2], which is nice but not nearly mature enough to use productively. As for the IronPython Visual Studio Add-on, my main complaint is about the complexity of the Visual Studio environment! I’ve even tried the VB.NET and Web Developer Express Editions and still find them not to be appropriate for occupational developers. Given that, it’s irrational to think that IronPython *within* Visual Studio Professional would be any better, especially since Scott Swaggart’s screen cast didn’t offer anything to change my mind.
>> our comment here doesn’t really make any sense:
>> "there is no intellisense for the .NET Framework"
>> Scripting language (including PHP, Python and Ruby)
>> typically don’t have intellisense for APIs, since
>> in a scripting world you often can’t infer the type.
>> Instead they usually have color coding of types and
>> keywords (which IronPython has) and sometimes syntax
>> completion.
First, please realize that those were not my comments; they were the comments of Scott Swaggart who prepared the IronPython screencast for Dr. Dobbs Journal. Maybe you should have actually read my post and reply before commenting?
Second, I disagree with your implied assertion that a "scripting language"[3] shouldn’t have Intellisense, especially since we are talking about IronPython which is hosted in the .NET Framework (I won’t attempt to debate the issue more broadly than IronPython within the .NET Framework at this point because it’s out of scope of the discussion, and also because I really don’t care enough to debate it.) But within the .NET Framework Intellisense can and should provide Intellisense for the .NET framework. Take the following code example (from Interactive Python Scripting Example [4]):
1. from System.Windows.Forms import *
2. from System.Drawing import *
3. f = Form(Text="Experiment #1")
4. f.ShowDialog()
There is no reason at all why IronPython shouldn’t be able to offer Intellisense after typing "from " on both lines #1 and #2. Further there’s no reason why if should be able to provide Intellisense after typing "f." on line #4.
Now I’ll freely admit that IronPython might not be able to offer Intellisense on lines #3 and #4 below, but there is no reason it shouldn’t be able to offer it on lines #1, #2, and #4 on the prior example.
1. def push(b, e)
2. global index
3. b.BackColor = colors[index]
4. b.Text = colors[index].Name
5. index += 1
>> IronPython works in the free Visual
>> Web Developer download
If you want to see recents issues I’ve had with Visual Web Developer, take a look at [5] and [6]. Suffice it to say that VWD still has a long way to go to meet the needs of the occupational developer (not the least of which is to offer and alternative web framework to ASP.NET. But more on that subject when I get time and my thoughts fully collected.)
BTW, please try both reading my posts and my comments in full considering their context before lecturing me (if you aren’t willing to read them in full, don’t lecture me.) Lecturing me only puts me on the defensive, and my best defense is a strong offense. That’s assuming you want to have a productive discussion instead of a flame war.
Now the further irony is that I’m very interested in IronPython, but it’s not ready for my use yet. It needs to be hosted outside of Visual Studio in a productive environment before I’m going to be really interested. Again, see[1] to better understand my concerns.
Also, I’d appreciate it if you’d include your full name when commenting and a reference to your blog so we can better know who the man behind the comments.
[1] http://www.mikeschinkel.com/blog/willmicrosoftmeetoccupationalprogrammersneeds/
[2] http://lynanda.com/mediawiki/index.php/Main_Page
[3] Though I believe Eric Lippert would make the distinction that Python is a "dynamic language" not a "scripting language."
[4] http://blogs.msdn.com/hugunin/archive/2004/08/23/219186.aspx
[5] http://forums.asp.net/thread/1508544.aspx
[6] http://forums.asp.net/thread/1508581.aspx
Scott wrote:
>> Mike’s assertion that .NET doesn’t have
>> scripting language support is just wrong.
Exactly when and how did I assert that .NET doesn’t have scripting language support? Quite the contrary, I think it does, and would be a great environment for scripting/dynamic languages. What I said that I wanted something productive, easy, and scalable. You really ought to *read* what I write before commenting.
Michael Foord wrote:
>> IronPython is an excellent language
>> implementation, which the Microsoft
>> team have used (amongst many other
>> things) to illustrate and improve
>> the Visual Studio support for
>> alternative languages.
>> I disagree with the Dobbs journal
>> article which says that IronPython
>> is not ready for the prime time, and
>> I think that to assert that as fact
>> without even trying the language is
>> ignorant. :-)
I will tell you the same thing I told Scott, my comments about IronPython were tangential to my point so please actually read my full post and comments to understand the context before calling me ignorant. :) I did point out in my comments that I should have referenced his Scott Swaggart’s screen cast in my comments about IronPython, but again my focus wasn’t on IronPython it was to point out that Eric didn’t address my prior posts main thesis. Ironically, you’ve now done the same.
As you’ll see in a more recent comment, I am interested in IronPython. But my biggest concern with IronPython is that it’s hosted in Visual Studio; please see [1] to learn more about those concerns.
>> We’ve found developing with IronPython
>> an excellent experience, and are
>> impressed with the Microsoft development
>> team. At last it *does* bring the
>> flexibility and development speed of
>> dynamic languages to the .NET platform,
>> which seems to be at least partly what
>> you are asking for.
You are experienced with Python and a professional development team. I am an occupational programmer. You are comparing apples to oranges.
Basically, because of your pro-IronPython perspective you are obfuscating my main points when defending IronPython because my main points had nothing to do with IronPython, they were about Microsoft needing to change it’s processes and release cycle, and also to address the occupational programmer. Ironically if Microsoft they were to do both as I envision it would only benefit IronPython so your raising a debate that distracts from my main thesis is really at cross-purposes for your own interests.
[1] http://www.mikeschinkel.com/blog/willmicrosoftmeetoccupationalprogrammersneeds/
I agree with a lot of what you say. It’s impossible for Microsoft to be closer to your needs than you are, which is a benefit of an open source platform. However, I don’t think you present the whole picture when you say this:
> And for a language that is used by a large number of people, the "expense" implementing the sugars are
> far outweighed by the expense to *not* implementing syntactic sugar when you look at productivity losses
> for the broader developer community including the bugs they need to fix for lack of the sugars that could
> otherwise have avoided.
You compare the expense of implementing a sugar to the expense of the loss of productivity by the developer community. But what about the expense of poorly implemented sugar to the broader developer community? That is also a potentially big expense.
As you know, everytime you have to stop to think about how something is supposed to work, you’re losing out on the flow. Even micro interruptions add up such as wondering whether F(++x) in C means x is incremented before or after the function call.
I think the small expense of that, added up over all developers who will ever use C, probably outweighs the loss of extra productivity that would’ve been gained by waiting another year to implement that sugar.
You mentioned that Microsoft is out of touch with developers in the field, but I don’t think such a blanket characterization can be made. First of all, you’re comparing their innovations with Ruby on Rails and Iron Python. But look at the developer market. These are early adopter technologies and a small part of the overall developer market. For this motivated market, there’s no way that Microsoft can keep up with open source products, because these very people are able to tweak open source to their own needs.
However there’s the much larger market of developers, working at established companies (some say boring), many of which forbid using open source toolkits. I’ve seen many companies that require a well known large vendor behind every technology they purchase. These are companies with legacy systems going back years that cannot afford a regression.
I would say that Microsoft is still capable of addressing real world problems. Just not YOUR real world problems. So they will compete fine for a long time, until you see legions of Morts moving over to Ruby on Rails. The question is, will they be exciting?
If Microsoft wants to target both groups, they really should look at embracing open source more. Forget Sandcastle, build on NDoc. Build on NAnt (probably too late now). Etc…
Haacked:
Thanks for your comments (Great gravatar, btw! :)
>>>"You compare the expense of implementing a sugar to the expense of the loss of productivity by the developer community. But what about the expense of poorly implemented sugar to the broader developer community? That is also a potentially big expense."
I’ll give you that that can be true.
>>>"As you know, everytime you have to stop to think about how something is supposed to work, you’re losing out on the flow. Even micro interruptions add up such as wondering whether F(++x) in C means x is incremented before or after the function call.
However, your argument is the same one people always use as an argument against syntactic sugar; i.e. that pre and post increment operators can cause problems. Unfortunately using pre and post increment operators as blanket justification against syntactic sugar is unfair as they represent one of the thornier isses related to syntactic sugar.
Further, I would definitely agree with you that syntactic sugar in *expressions* needs to be more strongly considered than syntactic sugar in *statements*. Contemplating syntactic sugar in expressions must consider as expressions use cases, yet syntactic sugar in statements need only consider the parsing of that specific statement. I myself would be much more cautious to push for syntactic sugar in expressions, but I think syntactic sugar in statements can be invaluable, especially when wrapping a well understood and many lines long design pattern into a single statement.
Thus it is with statements where I see the most benefit of syntactic sugar and the fewest potential problems.
As an aside, I cut my programming teeth on a language called Clipper that had a complex command structure (Clipper used the term "commands" for things that we might use for function calls.) In Clipper 5.0 they added "User Defined Commands" (UDCs) which allowed developers to create their own commands and since I wrote a book on Clipper 5.0, I learned them in depth.
The architects of Clipper REALLY understood the value of syntactic sugar and empowering developers to create their own commands really added to developer productivity. The downside of UDCs was they were really fragile because they were implemented via preprocessor.
What I’d love to see in VB.NET would be "User Defined Statements" implemented in the compiler making them far less fragile then Clipper’s UDCs. Imagine, given the example from above, if VB.NET had a "Statement" statement that would let a developer define his own syntactic sugar; Wow!
Begin Statement
Property _{prop} Into {prop}
Expands To
Private _{prop}
Property {prop}
Get
Return _{prop}
End Get
Set(ByVal value)
_{prop}= value
End Set
End Property
End Statement
>>>"I think the small expense of that, added up over all developers who will ever use C, probably outweighs the loss of extra productivity that would’ve been gained by waiting another year to implement that sugar."
If it only took a year to see these changes, I wouldn’t be complaining. But the reality it is taking 5 years, 10 years, or more and that assumes it will ever happen before VB.NET is no longer relevent. .NET is on it’s third revision after almost a decade on the market, yet many issues are still not addressed.
If we saw new issues addressed annually I’d be estatic!
Look at the third party like /n software, ComponentOne, and Infragistics; they release new versions every four months. Why does it take Microsoft approximately nine (9) times as long to release new things on their core developer platform? The answer is obviously that their process doesn’t allow them to do any better, and their process is exactly where I’m proposing change.
>>>"You mentioned that Microsoft is out of touch with developers in the field, but I don’t think such a blanket characterization can be made."
True, but it is hard to editorialize and at the same time treat all nuances with 100% accuracy. :)
>>>"First of all, you’re comparing their innovations with Ruby on Rails and Iron Python. But look at the developer market. These are early adopter technologies and a small part of the overall developer market. For this motivated market, there’s no way that Microsoft can keep up with open source products, because these very people are able to tweak open source to their own needs."
I mostly disagree. I’ve been planning to blog at length about how they can do this, but I’ve not had the dedicated time that it will require but I still hope to get there soon. Suffice it to say my advice for Microsoft will be akin to that old saying "If you can’t beat ’em, join ’em."
>>>"However there’s the much larger market of developers, working at established companies (some say boring), many of which forbid using open source toolkits. I’ve seen many companies that require a well known large vendor behind every technology they purchase. These are companies with legacy systems going back years that cannot afford a regression."
Yes, I agree there is a larger market of developers, working at "established"[1] companies. And that market is being served reasonably well although the pace of change with open-source will eventually steer them away from Microsoft’s developer tools, and hence the Windows platform if Microsoft doesn’t adjust in the mean time.
Take a look at Sun; not exactly the type of company that "established" companies shy away from. They just open-sourced ALL their software. Are "established" companies going to shun J2EE now?
>>>"I would say that Microsoft is still capable of addressing real world problems. Just not YOUR real world problems."
I agree Microsoft is still capable of addressing real world problems but you mischaracterized my comments; I did not say they were incapable.
What I said that the Microsoft’s development staff is not using their technology to solve real world problems in the same way most developers are using them. The relevency is that most open-source developers gain much greater insight into the needs of real world developers by actually using the tools their create as other real world developers would.
What to do? Microsoft should put the people developing tools in the field periodically so they will actually have to use the tools in the same way that their customers have to. They should hire people who are working at Microsoft Consulting Services to come in and work on the tools. They should create customer advisory boards that have as much input on direction of the developer division as the internal architects have. If they did this, Microsoft’s developers tool implementors would realize how hard it is to handle many common tasks with their tools.
>>>"So they will compete fine for a long time, until you see legions of Morts moving over to Ruby on Rails."
Actually I do see legions of Morts moving over. But not just to Ruby on Rails but also to Python/Django, PHP, and others. It doesn’t matter to which platform Microsoft looses developers; it simply matters that they loose developers to other platforms.
Riddle me this; what percentage of sites on the web use PHP vs. ASP/ASP.NET? If you don’t already know, you should find out. And Microsoft should be really afraid of this as it is evidence of what I’m saying.
>>>"The question is, will they be exciting?"
I don’t understand what you are asking; who is "they?"
>>>"If Microsoft wants to target both groups, they really should look at embracing open source more. Forget Sandcastle, build on NDoc. Build on NAnt (probably too late now). Etc…"
You hit the nail on the head. That’s the first thing you said in your reply that I agree with 110%! :)
Thanks again for your well-considered comments.
[1] Can I substitute "medium to large" here?
Hi Mike,
Thoroughly interesting post. There were a number of things here that really struck me as interesting, and I have to admit that I agree with you on your synopsis. Not only that, but your insight into group dynamics and the obvious issues between brilliant people working at the same company (it’s true: they do start to focus on their debate skills, and a number of internal issues that have nothing to do with everyone else.) This is a problem in many company’s, and it seems as if the only solution is to stay small (why are Google not suffering from this problem right now?) I think there are other solutions – here is not where to discuss them. However, it starts with the company and those involved to admit there’s a problem : and this, in itself, is certainly not easy.