This entry is cross-posted from my CodeBetter Community blog. The original post can be found there. My new feed address is:
http://codebetter.com/blogs/scott.bellware/rss.aspx

Test-Driven Development's "Red, Green, Refactor" mantra is a mnemonic shorthand for TDD’s development process.

Following Test-Driven Development, a developer first writes a failing test, then makes the test pass, and then writes the functional code that the assertions of the test are meant to lock in.

This is an overly simplistic view of TDD, but it's a level of understanding that many folks never really see beyond.  If you stop at such a shallow understanding of TDD, you'll probably end up scratching your head and wondering what all the fuss is about.  Again, "Red, Green, Refactor" is a mnemonic.  The richness of TDD isn't in the mnemonic, it's in the details.

Here is the TDD development process at a glance as oft-quoted in the software development community at large:

1. Write a test

• Think about how you would like the operation in your mind to appear in code
• Invent the API you wish you had (in case you missed it, this is software design, not software testing)
• Include all the elements in the story that you imagine will be necessary to calculate the right answers
• Write the test to fail

2. Make it run

• Quickly getting the test to pass dominates everything else
• If a clean simple solution is obvious, type it in
• If the clean, simple solution is obvious but will take a minute, make a note of it and get back to the main problem – making the test pass
• Quick test success excuses all sins, but only for the moment...

3. Make it right

• Now that the system is behaving, put the sinful ways of the recent past behind you
• “Step back onto the straight and narrow path of software righteousness” (Beck)
• Refactor (remove duplication)

"Red, Green, Refactor" is analogous to the The Karate Kid's, "wax on, wax off".

in the movie, "The karate Kid", Mr. Miyagi drills some karate moves into Daniel Larusso's muscle memory by finding a way for Daniel to repeat the movements without focusing on the fact that he's "doing" karate.  At the time, Daniel thinks that his new mentor is simply taking advantage of him as free labor until Miyagi shows Daniel that the "wax on, wax off" motions are in fact the motions for blocking an attack.  Miyagi habituated Daniel to the motions by having him use the motions to wax Miyagi's small fleet of classic cars.

There's more to "Red, Green, Refactor" than meets the eye, just as there is more to "wax on, wax off" than meets the eye.  The question is whether the petitioner has the discipline to drill the practice until it is second nature.

TDD is a practice.  It's something that requires some drilling at the outset.  TDD isn't a tool, and there's no magic want that a teacher can wave over your head to transfer practice and knowledge into you.  TDD is software Kung Foo.  You're not going to get it by reading comic books or and sitting vegetatively on your ass in front of an XBox.  On the upside, it'll take much less time to become proficient in TDD than to become proficient with Miagi Do Karate (not considering the karate kid's ability to become a champion fighter in the spaces of a few weeks).

At first, exercising the TDD process will take up a lot of attention.  Developers spend a lot of attention on keeping to the test-first way of doing things, and little attention is spent on becoming aware of the changes to software deign that occur by engaging in test-first programming.

At this stage, TDD practice may seem as meaningful and fruitful as waxing cars did to Daniel's karate training.  You might be tempted to quit, and it's at this stage that most undisciplined developers abandon their TDD practice and abandon themselves to their previous pathetic programming proclivities.

Part of penetrating the seeming frivolousness of Test-Driven Development's practice is getting to an understanding of why we start with a failing test, and follow up the failing test that doesn't really validate the code in any serious way.

It's not valuable to simply accept the TDD dogma at face value and expect that you'll to keep at your practice faithfully until the "Red, Green, Refactor" lights come on.  There's a darn good reason for starting with a failing test, moving on to a simple passing test, and then getting down to writing real functional code.

Tests are instruments of measurement.  They measure the correctness of functional code and prove that functional code has no defects.  But there's an inherent problem with tests... they are made with the same raw material that functional code is made of and subsequently they are subject to the same defects.  Put another way, if you are using code to prove that some other code doesn't have any bugs, what's to say that the test code itself doesn't have any bugs?  How do you prove that the test code is defect-free?

You can write a defective tests just as easily as you can write defective code.  The tricky thing with defective code is that it's likely that you won't simply catch the defect by reading the code.  You know this to be true because you've often sat down to fix a defect and stared at it for some time before you realize what the defect is.  Defects can sit right under your nose and not even be detected.  Your ability to catch a defect can depend on all kinds of external influences - how much sleep you've had, how many distractions are in your workspace, the composition of your new allergy medication, etc.

It's so easy to write defective code, and thus defective test code, that TDD prescribes a disciplined process for writing tests that seeks to insulate you from the human frailties that are often the root cause of defective code to begin with, namely, programmer self-overestimation, and inattentiveness or distraction.

Measurement instruments often have to be calibrated before they can be used reliably.  My desktop scanner came with a white balance sheet.  The scanner's software can be calibrated to an objective measurement of the color white by scanning the white balance sheet.  Measurement instruments are calibrated to objective, known states.  This calibration is what we're doing by going through the Red and Green phases of the TDD development cycle.

By starting with a failing test, we are calibrating our software correctness measurement instrument, i.e.: our test, to a known failure state.  This proves that the test detects an invalid state in the software under test and correctly reports the failure.  We do this in the most simple way, often just returning a hard-coded invalid response from a method, or something equally simplistic.

Moving on to the green phase, we calibrate the test to a passing state.  This proves that the measurement instrument can correctly detect and report that the software under test is working within its expected design parameters when it is functioning correctly.  As with the failing test, we cause the test to pass with very simple code, often returning a hard-coded valid response from a method.

We never calibrate tests using real functional code.  Real functional code might pass or fail for reasons that aren't predictable.  For example, the functional code may have a defect.  If you calibrate test code against defective functional code, you haven't actually calibrated you test at all.  In fact, if you calibrate against defective functional code, you risk introducing even more defects into your codebase since you will believe that the calibrated test is in fact a reliable measurement instrument when in fact, it won't provide you with reliable measurements at all.  I don't calibrate the white balance of my scanner by feeding it a page of my local newspaper since this would give the scanner incorrect information to base its color offset algorithm on.

We need to start the refactoring phase of the TDD cycle with reliable, calibrated tests because we are ultimately going to be incrementally introducing small changes into the codebase and with each small change we will execute the calibrated test to make sure that we are still on track.  We follow this process whether we're introducing a new bit of code, or if we're introducing a change to some existing code.  The process is always the same - create an unquestionably calibrated test, and then introduce the functional code in small increments.

Many small increments of change are used because it's easier to tell at any stage whether the last bit of code you wrote broke the code.  When a test fails, we fix the code immediately.  If you introduce change in big chunks, it's harder to figure out exactly which new bit of code from that larger chuck of work is actually responsible for breaking the code.  When you code in large chunks, you end up spending more time coding as you'll spend more time figuring out what went wrong when something goes wrong.  Invariably, you'll need to use the debugger to figure this out. Developers with a TDD practice spend much less time debugging code, and much more time writing code and refactoring code toward its inherent, optimal design.

Debugging code is a slow, time consuming process.  Time spent in a debugger is sloth time.  You might be thinking that you're perfectly effective in a debugger and that you don't have any objections to doing code validation in a debugger rather than in a well-factored unit test.  This is merely an assumption fed by how habituated you are to using a debugger.  Without having a TDD practice, you have no basis of comparison for how ineffective debugging is compared to writing well-factored unit tests for well-factored code.

Arriving at a solid understanding of why TDD insists on starting with a failed test, and then making the test pass will go a long way to helping you internalize the necessity of the TDD process.  The unit test is the force driving the design of your functional code; or the "factoring" of your functional code.  The unit test must be as defect-proof a possible before we put our trust in it as the single most influential aspect of our coding and detailed software design practice.

Understanding why we calibrate the test doesn't explain how to calibrate the test, and how to do so effectively.

You can still do things during the calibration of your test that by and large don't really serve to calibrate the test as much as they do to provide you with an illusion of having a calibrated test.

For example, you can fail a test by throwing an exception from the method under test.  If the method under test is never expected to throw an exception, then this would be an entirely inappropriate way to calibrate the  test.

The first part of the TDD cycle where a failing test is written should be stated, "write a meaningfully failed test," or "fail the test meaningfully."

If a method is expected to return an integer that is greater than 0, then fail the test by having the method under test return -1.  Returning -1 from the functional method is in keeping with actual functional behavior that would be considered invalid and that the test should detect and report as a failure.  Throwing an exception from the method under test will indeed cause the test to fail, but not in a way that is within the design expectations of the method, and therefore the test won't have actually been correctly calibrated toward detecting a real invalid state.

With this in mind, here is a restatement of the first step of the TDD process, slightly modified to disambiguate the knowledge that TDD practitioners often take for granted:

1. Write a test

• Think about how you would like the operation in your mind to appear in code
• Invent the API you wish you had
• Include all the elements in the story that you imagine will be necessary to calculate the right answers
• Write the test to fail meaningfully

"Now use head for something other than target" - Kesuke Miyagi

This entry is cross-posted from my CodeBetter Community blog. The original post can be found there. My new feed address is:
http://codebetter.com/blogs/scott.bellware/rss.aspx

The poop has hit the propeller for Microsoft over the past few days in regards to its Guidelines for Test-Driven Development article published on its Microsoft Developer Network portal (MSDN).

Test-Driven Development (TDD) is a well-defined software design method that focuses on facilitating testability through loose coupling. The loose coupling arrived at by TDD fosters better class factoring and subsequently enables vastly improved opportunities for reuse harvesting. In addition, Test-Driven Development teaches software developers who are relatively weak in object-oriented (OO) programming and design how to use OO techniques and provides clear guidance for the employment of design patterns. I can’t imagine another method that offers as much to software developers as TDD does in terms of effective design, extraordinary increases in software quality, and OO learning.

On Test-Driven Development and Agile, Microsoft seems to have has lost its way, lost its courage, and lost its ability to innovate. By missing the Agile boat, Microsoft is missing a key opportunity to transition millions of legacy developers to the .NET platform through the enhanced OO learning offered to developers who engage in Agile development practices. Microsoft is missing even more opportunities to rapidly adapt its software development tools to emerging requirements in contemporary software development because Microsoft isn’t yet harnessing the power of the adaptive design and delivery enjoyed by those developers and shops that engage in Agile software development.

Faced with a critical decision of innovation over stagnation, Microsoft has cowered into the old familiar waterfall approaches that ensure that its customers are capable only of repeatedly investing money hand over fist into efforts that produce utterly disposable software. The practice guidance for Visual Studio 2005 reeks of outdated software development practices, and the Guidelines for TDD article is a shining example of how smelly things can really get.

A clear distinction needs to be drawn between mere test-first programming and Test-Driven Development. This is a point that is largely lost on most developers new to TDD – and by “developers new to TDD” I mean those for whom the TDD lights still haven’t come on. For me this period of darkness lasted for over a year after I had read Kent Beck’s book on the subject. For over a year, I had assumed that I knew what TDD is and I had assumed that I was practicing it. Imagine my surprise when I realized that there was more – much more – to TDD than I had assumed.

Reading the main Test-Driven Development forum on Yahoo Groups, there was a lot of talk on a lot of esoteric software stuff that was way over my head. Originally, I thought that all this esoteric stuff was just stuff that applied to Java developers, since at the time, most of the TDD people were Java developers along with a smattering of people using C++. Over time, it became apparent to me that these esoteric issues in software design were essential aspects of object oriented design and are equally applicable to .NET, and it was through TDD that I learned that I had totally overestimated my knowledge of OO.

It was also through TDD practice that I learned one heck of a lot about OO, design patterns, component design, and application design and architecture. TDD practice puts learning in high gear and keeps it there. Of course, if you don’t think that learning is part of your job as a software developer, then TDD is probably not for you. And this might be the message that Microsoft is sending out regarding the kind of software people that far too many .NET developers have devolved into.

Rather than shape up Visual Studio to be amenable to Test-Driven Development, and in its ignorance of the nature of Test-Driven Development, Microsoft’s marketing people have simply tried to snow job the developer community into swallowing a wholly different definition of Test-Driven Development. And the very text outlining Microsoft’s understanding of Test-Driven Development betrays its almost complete ignorance of why Test-Driven Development is effective and why it is practiced in the way that it is.

So, this is for the folks at MSDN. Listen up… you’ve already embarrassed yourself once, and you’ve embarrassed your insider and MVP communities by association this time as well. So here’s more of the skinny on TDD:

TDD is a software design methodology. It’s not a software testing methodology. As such, a guidelines for TDD document doesn’t belong in the MSDN content tree for VSTS for Testers – it’s a topic for developers. TDD arrives at its design values by considering client code first before considering server code. It uses tests to act as clients because it’s vastly cheaper and faster to code a test method than to build an entire visual application to incrementally implement and validate a design intention. Because tests are used as client code in the process of client-driven design, we arrive at the end of any given development cycle with tests in hand. And in using tests as clients, we learn a lot more about the needs of testability, and how to factor our class libraries so that internals that would traditionally be hidden way down in the dark, buried branches of code are surfaced through design patterns and exposed directly to tests. This in turn leads to more simple test classes and thus to greatly facilitated maintenance. The smaller class factoring and the structural decoupling of these smaller classes via interfaces (and sometimes abstract classes) allow these assets to be more easily harvested for reuse. And the emergence of the justification for OO practices like polymorphism and adaptation demonstrates to even the most novice OO developer where and why to use design patterns, and which design patterns to use.

Oh yeah, and because we base our tests on a well-known test frameworks like those in the xUnit family (like VSTS), we also use our design scaffolding as a regression test suite, further ensuring the sustainability of our codebases.

Testing is a side effect. Design is the goal of TDD, design for testability flows naturally from there, followed by de-coupling, and reusability. TDD is a king among contemporary software design practices. It delivers on so many measures of software development goodness.

You simply cannot achieve the aims of Test-Driven Development if implement sweeping assumptions of the structure of classes before you set down the kind of code that incrementally validates your assumptions.

If a developer leaps to conclusions about design of a class or family of classes, he will have missed the optimal design of the class. As the implementation of a class deviates from its inherent optimal design, implementation and maintenance costs for that class increase exponentially as the effort to cajole misshapen classes together into a system of misshapen classes increases. This increase in cost an effort is compounded by the number of classes in a system. Most codebases can’t sustain this pressure for too long and they are often disposed of and re-written entirely from scratch – using the exact same development methodologies that caused them to degenerate into entropy to begin with.

In TDD, you start with an understanding of what you need because scenario-based customer specification through stories or use cases tells you what you need. You write down a set of things that the software should do to satisfy a story, and you have the cajones to admit to yourself these things that you have written down are purely conjecture until you get them into code. We know that these things are conjecture because every developer learns about what he is coding from the first few lines of code he writes. As he writes more code, more is learned and previous assumptions are either ratified or disqualified and the code is morphed to address new understandings. We don’t waste time on detailed design using any tool other than code because code is ultimately the only thing that can validate detailed design. Committing to the pursuit of detailed design in anything other than code leads to waste. Those design artifacts are ultimately invalidated by code and the artifacts are abandoned without ever having delivered sufficient ROI.

To quote Mary Poppendieck, “Design is a knowledge-generating process,” and because knowledge is continually generated during coding, then coding, as Jack Reeves pointed out long ago, is a design activity.

Coders are making design decisions with every method they implement. They do design whether they use Test-Driven Development approaches or not. Evolutionary design is the way that software is created. It’s endemic to the medium. Software development is unlike any other kind of construction activity in this regard, and software development is like most forms of product development in this regard.

TDD simply supports the reality of software creation with a process and a set of techniques that are wholly optimized for the reality of creating software. Visual Studio on the other hand, is optimized for the reality that customers are still willing to believe in material solutions to problems that are essentially intellectual in nature, and Microsoft marketing folks are right there to sell snake oil to the gullible.

Should Microsoft be held responsible for selling lack-luster products into a market that is snuggly snoozing in a profound yet comforting sleep of epistemological ignorance? It damn well should be held responsible if it is not only going to sell software development tools, but also prop itself on a high horse and hold itself up in the community as a provider of software development practice guidance. If Microsoft cannot be trusted to do both and to do so with deep convictions to truth in advertising, then it needs to return to being merely a manufacturing company.

Ironically, the Agile development approaches that Microsoft so obviously eschews would have provided the company with the techniques that would have enabled it to provide support for Test-Driven Development in its own software development tools. Without the influence of Agile development techniques, Microsoft is doomed to continue to engage in building massive, monolithic, tightly-coupled products like Visual Studio that cannot be quickly adapted to meet emerging market pressures.

In the case of Test-Driven Development, the unit testing tools available in VSTS fall short of the mark, and leave the NUnit framework and Jamie Cansdale’s Test-Driven.NET plugin for Visual Studio 2003 and 2005 to rule the roost as the leaders in tooling for Test-Driven Development for Visual Studio. That’s not to say that Test-Driven Development isn’t an interest for Microsoft. However, Microsoft’s lack of foresight in identifying the need to support TDD in Visual Studio early enough in its Visual Studio 2005 product development cycle and its lack of agility in adapting to emerging requirements for tools to provide direct support for TDD once it had become aware of the need to support TDD have conspired to leave Microsoft little recourse at this late date.

In fact, members of Microsoft’s insider community repeatedly asked for support for Test-Driven Development in Visual Studio 2005, and Microsoft was repeatedly publicly criticized for ignoring TDD requirements throughout its long beta and release candidate phase leading up to the RTM release of Visual Studio 2005 in early November. Microsoft is so utterly un-agile, that it isn’t even able to respond to customer requirements that it had known about for a year and a half. And this is a company that purports to provide Agile development guidance to the software development community at large?

Ultimately, Microsoft caught on to the need to have a Test-Driven Development story for Visual Studio 2005 when it had already become too late for them to respond to the requirements. Well, too late for waterfall, anyway. So, at some point in recent history, Microsoft marketing folks made a bet that if they simply redefined Test-Driven Development so that it matched the existing tooling provided by Visual Studio 2005, they would be able to claim support for tooling requirements for contemporary software development techniques and approaches without actually having to worry about actually having that support in their tools.

Going with the assumption that the software development community at large is as ignorant and out-of-touch as the folks in Microsoft developer tools marketing in regards to contemporary software development, they simply posted the atrocious “Guidelines for Test-Driven Development” on MSDN.

Guidance provided in this article fundamentally contradicts some of the most essential and elemental practices and goals of TDD. If this guidance were to be put into practice, the practitioners would never glean the benefits promised by TDD. Because the guidance commits to neither waterfall nor to TDD, its fragmented nature almost guarantees that a practitioner of these guidelines be doomed to software hell.

In waterfall-style development, a developer starts with sweeping assumptions about a class’s design and might capture them in a visual documentation tool like a class designer. If the developer were test-oriented (not Test-Driven mind you, but test-oriented), he might then code some test stubs. If he were using a powerful tool, the tool might generate the tests based on the code generated from the class design.

However, in doing so he would lock in his design at a point that is far too early in the development process. Wrapping so much test harness around so many non-validated design assumptions would likely cause those design assumptions to be fixed into place. Even if the developer had wanted to incorporate new knowledge into the design, there would be so many dependencies at such an early stage in the development that he might just as soon live with the junk he had just created.

This fixing of design assumptions in place too early in the process is one of the weaknesses of waterfall that TDD addresses quite effectively by providing practices that reinforce a very granular approach to incremental design and implementation. The approach is granular enough to allow understandings about the code under development to be taken into consideration immediately when these understandings emerge.

Microsoft’s guidelines specifically recommend an approach that ensures that the failures of traditional software development will be upheld going forward into the age of VSTS:

Step 6. Define the interfaces and classes for your feature or requirement. You can add a minimum of code, just enough to compile. Consider using the Class Designer to follow this step.

Step 7. Generate tests from your interfaces and classes.

A more ignorant approach to TDD would have been hard to come up with. These steps in Microsoft’s TDD guidance essentially describe nothing short of an anti-TDD process, and the whole document has the nerve to package up the guidance as factual TDD guidance. It’s truly a shameful example of the worst product marketing behavior.

Microsoft has so overestimated its own understanding of TDD and the developer community that it has left itself open to criticism from some of the most notable figures in Agile development.

Robert Martin has this to say:

Microsoft should pull these guidelines and try to figure out some way that the tool might actually support TDD. Redefining TDD as some kind of waterfall, just because their tool supports it that way, is not very helpful to the industry; and most programmers are all to clearly aware of this. If Microsoft would like to win the hearts and minds of more developers, they'd better try to figure out where the industry is actually going, rather than trying to drag the industry into it's tool. IMHO they should look very carefully at Eclipse, and IntelliJ -- especially IntelliJ.

Bottom line: We don't need tools that help us do waterfall better. We don't need vacuous process guidlines with 14 linear steps describing something that is not TDD while claiming otherwise. We don't need to be force fed Visual Studio. What we need from Microsoft is for them to stop talking and listen.

And from Michael Feathers:

The style of TDD described in the guidelines would have us jump ahead and write five, ten, maybe twenty test cases before getting the first one to pass. You can do that, but it's like putting on a set of blinders. When you don't have to formulate the next test case right after passing the last one, there isn't much chance to think about improvements. Worse, there is a disincentive to thinking about them: if you find any, you have to delete all of the speculation you've hardcoded in the tests and interfaces you created in advance.

Right now, I hope that MS revisits their guidelines. Yes, they have a tool that generates test stubs, but using it for TDD looks counter-productive. Adding stubs to legacy code? That would be great, but getting in the way of the feedback cycle? Bad. Bad. Bad.

From Ron Jeffries:

The original article, at msdn2.microsoft.com/en-us/library/ms182521.aspx, describes TDD as a process where you get all your requirements understood, then define all your tests, then design an object's full interface, implement the tests, and make them work. It then goes on to explain how all these cool features in VSTS help with this "TDD" process.

The features might be good, but the process described is not TDD, nor a reasonable variation thereof. Test-Driven Development was clearly defined by Kent Beck, and has been described by others, such as Dave Astels, and even myself. It is a process where the tests are written one at a time (though one might make note of some possible tests for the future), and the tests are used to help define the design and develop the code. The Microsoft version of TDD is indistinguishable from a single-object waterfall model, to a first approximation.

Someone more paranoid than I am might conclude that Microsoft is intentionally trying to co-opt a perfectly good practice, pervert it, and tie it into mandatory use of their tools, which are, at this writing, about half as good as what's available in Open Source and commercial plug-ins for Visual Studio. I prefer to assume that it's just ignorance on their part, but I'm prepared to change my mind.

The article is shameful. If it isn't intentionally malicious, it is at least ignorant and ill-conceived. It should be taken down and replaced with something that correctly reflects industry usage of the TDD terminology.

From Sam Gentile:

The Microsoft Guidelines for TDD are creating quite a storm of protest from people in the Agile community for very good reason. They don't at all describe Test-Driven Development (TDD). They got it all wrong.

From Jeremy Miller:

No, no, no. Autogeneration of the unit tests from interfaces or classes might sound nice, but it isn’t really correct and might not be all that useful. It’s also not Test Driven Development. TDD is using unit tests to drive the design of the code. One of the “crossing the Rubicon” moments in learning TDD is when you learn to define the interface of a class inside the unit test. That’s right, use the unit test to define the intended function of a new piece of code, and then write the code to make the unit test compile and pass (ReSharper will very happily create the method stub for you). I still do some paper and pencil work before coding, but by and large I’ve found that I have better results when you are determining a class’s signature while writing the unit tests. Writing unit tests first will inevitably simplify the usage of a class (because you’ll want to minimize the mechanics of creating the unit tests). I certainly don't think that you throw away traditional design techniques when you do TDD, but you can't ever let the code and design get very far ahead of the unit tests.

Another thing to keep in mind with the autogeneration of unit tests is that unit tests do not map one to one with methods and properties. You don’t test methods one by one. You test logical pathways through the code.

One of the key things to success in TDD is to very deliberately design for testability. The best way to accomplish testability is to write code “Test First”. If you design and create a lot of code before you switch to retroactively applying unit tests you’ll often find that the unit tests are hard to write. Also, you will probably not achieve the same level of code coverage that you would if you code test first. Actually, let me put this more strongly – retrofitting existing code with unit tests is hard. It’s perfectly acceptable to spend some time contemplating the design before coding, but do write the unit tests first. TDD will go so much better when you write the tests first.

My single biggest irritation with Microsoft’s guidance to .Net developers and their development tools is Microsoft often seems to ignore anything outside of the Redmond campus. The advice they’ve given is contrary to existing best practices for TDD that have resulted from experience. Patterns and Practices team, are you out there? I know you guys are using Agile methods because I read your blogs. Do y’all really buy off on the MS TDD “recommendations?” Can you do something about this?

A number of TDD and Agile community folks have chimed in on this issue on the home base of TDD community at Yahoo Groups, ultimately driving curious traffic to the guidelines document. Overwhelmingly, people who have taken the time to rate the document have given it the lowest possible rating. As Michael Feathers pointed out about the article’s rating on the TDD forum, it’s a “shame it's asymptotic to 1.0.”

On a related note, I will be teaching a free, all-day workshop on Test-Driven Development in Houston on Tuesday, November 29th. Ironically, the workshop will be held at The Microsoft Houston offices, but not to worry, I’ll be presenting actual TDD, not the MSDN stuff.

This entry is cross-posted from my CodeBetter Community blog. The original post can be found there. My new feed address is:
http://codebetter.com/blogs/scott.bellware/rss.aspx

From Microsoft's Guidelines for Test-Driven Development on MSDN, here’s what Microsoft is offering as Test-Driven Development through to the community of developers who are having a harder and harder time trusting that MSDN will be used to disseminate real practice guidance rather than marketing propaganda:

1. Define the requirements of your application.

2. Familiarize yourself with the feature areas of your application, and decide on a single feature, or the requirements of a feature, to work on.

3. Make a list of tests that will verify the requirements. A complete list of tests for a particular feature area describes the requirements of that feature area unambiguously and completely.

4. File work items for feature requirements and for the tests that need to be written.

5. In Visual Studio, create a project of the type you want. Visual Studio supplies the initial production code in the form of files such as Class1.cs, Program.cs, and Form1.cs, depending on the project type.

6. Define the interfaces and classes for your feature or requirement. You can add a minimum of code, just enough to compile. Consider using the Class Designer to follow this step. For more information, see Designing Classes and Types. Note , The traditional TDD process does not contain this step. Instead, it advises that you create tests first. This step is included here so that, while creating tests, you can take advantage of two features in Visual Studio 2005 Team System: the GUI design capabilities of the Class Designer, and the automatic test-generation capabilities of Team Edition for Testers.

7. Generate tests from your interfaces and classes.

8. Compare the tests that have been generated with the list of tests you prepared in step 3. Create any tests that are missing from the list you wrote in step 3.

9. Organize your tests into test lists. You can, for example, base your lists on test use, such as check-in tests, BVTs, and tests for a full-test pass; or on area, such as UI tests, business-logic tests, and data-tier tests.

10. Update the generated tests to make sure they exercise the code in ways that cover the requirements that you have defined.

11. Run your tests.

12. Verify that all your tests fail. If any test produces a result of Inconclusive, it means that you did not update the generated code to add the appropriate verification logic for that test. If any test produces a result of Passed, it means that you did not implement the test correctly; this is because you have not implemented your production code, so no tests should pass.

13. Implement the interfaces and classes of your production code.

14. Run your tests again. If a test still fails, update your production code to respond. Repeat until all the tests pass.

15. Pick the next feature or requirement to work on and repeat these steps

1. Define the requirements of your application.

Or, you could allow your customers to define the requirements of your application, but if you feel that you could do a better job, go right ahead. Certainly many developers have defined requirements in the past on many failed projects – I guess there’s no need to buck that trend now.

2. Familiarize yourself with the feature areas of your application, and decide on a single feature, or the requirements of a feature, to work on.

Sweet. Now you’re talking my language. You’ve probably read through the story cards that the customer has told you were most important for the iteration – unless of course you’re also making that decision as well as defining the requirements. Sorry for asking, but is there an actual customer involved in this project? It's ok if not - everyone builds themselves little dev tools every now and then acting as their own customer, but try to get real requirements from real customers when doing real customer projects, and try to talk to real customers when setting the priority of development.

3. Make a list of tests that will verify the requirements. A complete list of tests for a particular feature area describes the requirements of that feature area unambiguously and completely.

It's not the list of tests for a particular feature area that describes the requirements of that feature, it's the executable tests themselves. Valuing executable code and team communication over documentation is a fundamental tenet of Agile development.  The test list will go out of date in time, but the tests remain current to the system under test. When it comes right down to it, the tests provide a more accurate depiction of the requirements.

Ultimately, assuming that you could write a complete set of tests or a list of tests for a given feature all at once assumes that you can envision the complete set of classes, as well as their internals and API’s before you write a single line of code. Don't assume or expect that your test list of that your collection of tests will be complete - especially not at the outset. Since having a complete list is impossible, staying within the bounds of reality is often a much more comfortable place to be. Just start with the tests that you can conceive of, and understand that you can add to this list when the need presents itself. Know that any requirement will never be described unambiguously and completely – not even if described using Visual Studio’s work item tracker.

I heard that out of the box, the work item tracker doesn't actually track requirements. I must have got that wrong. I mean, what would you assign tasks to if not stories (or use cases if doing MSF or RUP, etc).

4. File work items for feature requirements and for the tests that need to be written.

Doh! Spoke too soon.

By the time you're finished filing work items for tests, you could have started coding tests and prove that a non-trivial portion of the tests you filed aren't relevant. Write `em down on a piece of paper, put `em in OneNote, or write `em in todo comments in the code itself.

Somewhere in the course of implementing the first half of the tests on your list, you will likely have already invalidated some of your assumptions about the code you’re writing and you’ll need to change the test list and possibly some of the code you’ve already written. The items on the test list are transient, and often rapidly so. Don’t put anything down in any medium that isn’t as quick to amend as pencil and paper if you don’t expect it to be permanent.

I can't see how using the work item tracker for tests is anything that I'd think of as Agile.  Maybe if it were little "a" agile, like "agile", as in "MSF for agile".  Yeah, that does it... not really "Agile", but "agile", or "agile-ish".

By the way, do you have rights to delete work items from the work item list once they’ve been added?  When you start out doing TDD, you're invariably going to list tests whose need will be invalidated as you proceed through the TDD development process.  You're going to need rights to delete work items when you realize that your up-front assumptions about all those tests turned out to be off the mark and need to be deleted. Most work item tracking installations I've seen don't offer that privilege to developers.  Make sure the CMM mafia at your company will allow you the appropriate rights.

5. In Visual Studio, create a project of the type you want. Visual Studio supplies the initial production code in the form of files such as Class1.cs, Program.cs, and Form1.cs, depending on the project type.

Unless you’ve already done so… this step would seem to suggest that you need to create a project every time you write a test. I'm sure that's not what's being suggested here, but just to be on the safe side, just create a test project… that’s always a good place to start. Maybe you'll need more than one at some point, but you'll know when you'll need another one when that need arises. One will likely be sufficient at the outset.

6. Define the interfaces and classes for your feature or requirement. You can add a minimum of code, just enough to compile. Consider using the Class Designer to follow this step. For more information, see Designing Classes and Types. Note , The traditional TDD process does not contain this step. Instead, it advises that you create tests first. This step is included here so that, while creating tests, you can take advantage of two features in Visual Studio 2005 Team System: the GUI design capabilities of the Class Designer, and the automatic test-generation capabilities of Team Edition for Testers.

So... fundamentally avoid the key enabling technique in Test-Driven Development so as to make use of IDE features that support development techniques and processes from bygone years.

Don’t think for a minute that code completion isn’t a feature of the Java IDE’s where TDD was born. And didn’t I see code completion in Visual Studio .NET 2003? I’m sure I used something called intelliSense when I was doing TDD in VS 2003.

TDD isn't about taking advantage of every IDE feature at every moment, it’s about defeating the risks to projects that traditional development approaches practically ensure. Don't trade the benefits of TDD for a quick fix from IntelliSense. Code completion is still a great thing, and it will come into play as soon as you're through the initial design phase of TDD. Don't expect IntelliSense on something that doesn't yet exist, and don't bring that something into existence prematurely just to justify the existence of code completion.

There is no such thing as traditional Test-Driven Development. This is just a bunch of marketing clap-trap aimed at rationalizing certain VS 2005 features into TDD that have little to do with TDD. The sidebar could have been phrased, “The TDD process does not contain this step. Instead, it advises that you create tests first. This step is included here so that, while creating tests, you can take advantage of the antiquated, waterfall approaches to software development that have given rise to the catastrophic rates of software project failure that our tools are built to support. Please use our tools the way our product designers have intended them to be used. They’re so close to retirement and we just don’t want to have to ask them to think new thoughts at this late stage in their careers. They’ve worked so hard and they deserve a rest. Maybe you could stomach a little more software project failure until they’ve moved on. By the way the automatic test-generation capabilities are only available in Team Edition for Testers so please pay an extra five thousand dollars if you’re a developer and you mistakenly purchased Team Edition for Testers.”

7. Generate tests from your interfaces and classes.

Riiiiiight. Even Microosft's own Jack Green questions whether code generation and Agile development is reconcilable in the opening pages of his Software Factories book. Yo Microsoft, you’re reading your own books, aren’t you? Well, if you were, I expect you'd have read Newkirk's MS Press book and Jeffries MS Press book and we might not be in this mess.

You can only have your cake and eat it too if and only if you also happen to be drinking the special Kool-Aid. Well maybe you don't actually read books about software development. I can buy that... since these guidelines seem targeted at people who don't actually study in their chosen field of endeavor.

Generating tests from existing interfaces means that without a doubt, you are not doing Test-Driven Development.  Test-Driven Development is an adaptive software design methodology that evolves your classes and interfaces into existence by setting hard expectations for them in client code. In the case of TDD, that client code also happens to be test code. Tests are a side-effect of TDD.

This isn't about testing, it's about design. If you've already got classes and interfaces, you've already missed the moment in the game where TDD brings its power and its benefit to bear on your project or product.

8. Compare the tests that have been generated with the list of tests you prepared in step 3. Create any tests that are missing from the list you wrote in step 3.

Why not just go one test at a time, tests-first, so you don’t need to protect yourself from oversights that you don’t have to make to begin with.

9. Organize your tests into test lists. You can, for example, base your lists on test use, such as check-in tests, BVTs, and tests for a full-test pass; or on area, such as UI tests, business-logic tests, and data-tier tests.

I really don't know what this is all about.

I personally like to keep have separate test assemblies for tests that are targeted at concerns that are different enough to be distinguished by separate lists. At least then, I can choose to execute them separately in automated builds and know which system concerns have failed tests. Maybe that's what these test lists are for. Dunno.

10. Update the generated tests to make sure they exercise the code in ways that cover the requirements that you have defined.

All at once, or one at a time?  Three at a time?  Five at a time.  What are the guidelines here?

If I need to add new tests, should I update the work items in Team Foundation each time a new test occurs to me, or just throw it on my todo list that I keep beside me for just these occasions? If not, do I just go ahead and code up a mess of stubs for the missing tests? If I’m not using TFS, should I add the tests to my list, or just add them to my test class?

Surely you thought this one through while speculating on what Test-Driven Development might be when Microsoft marketing asked you to write these guidelines that you felt were too sweet of a fictional writing gig to give up even though you’ve never actually done any Test-Driven Development.

11. Run your tests.

Ok. I guess this is a good a time as any. I can’t really see where this process is heading, so I might as well just go along for the ride at this point. If I were actually doing TDD, I'd have already had feedback from at least one test by this point and I'd already be well into validating my design assumptions.

12. Verify that all your tests fail. If any test produces a result of Inconclusive, it means that you did not update the generated code to add the appropriate verification logic for that test. If any test produces a result of Passed, it means that you did not implement the test correctly; this is because you have not implemented your production code, so no tests should pass.

Are you suggesting that I write a whole bunch of functional code in one gulp to fail each one of the generated tests?

Failing a test in TDD means meaningfully failing the test in a way that the functional code might actually fail a test. It doesn't mean just throwing a NotImplementedException or executing VSTS's assinine Assert.Inconclusive.

To meaningfully fail each unit test in the Red phase of TDD’s Red-Green-Refactor process, I’d have to know about meaningful failures for each public API element of the class under design. Since I haven’t really been able to validate the design yet, and knowing the design will absolutely change once I start implementing it, further causing me to drop or change the tests that I’ve identified, won’t this cause me to write a pile of code that I’ll invariably have to throw away later?

Why not just go one test at a time, and one public API element at a time and minimize the waste by adapting to new understandings as those understanding occurs. The approach forwarded by this document ensures that I’ll produce the volumes of waste that, well, waterfall has always ensured.

I would bet you, the author, would have a wholly different understanding of TDD if you actually practiced it. What the entire Test-Driven Development community knows, and that you have bet against, is that it is painfully obvious that you are not actually a TDD practitioner, and yet here you are giving practice guidance on TDD. I can't wait for your book on thoracic surgery. I hope that particular set of practice guidance doesn't produce the kind of failures that the TDD guidance ensures.

13. Implement the interfaces and classes of your production code.

Yep. Here’s where I’m gonna find out that a pile of the work that I just did in the previous step was unnecessary.

Well that's not entirely true. There's an even worse outcome...

Since I’ve just spent a pile of time writing code to meaningfully fail those tests and I’m quite likely to have become sensitized and predisposed to the design expressed in that code, I'm more just as likely to cajole my software to fit the interface that I've prematurely committed to. There goes TDD’s promise of clean design from the start, eh? Ah waterfall, my old friend. How I’ve missed you. How's your old pal MSF?

14. Run your tests again. If a test still fails, update your production code to respond. Repeat until all the tests pass.

I can’t imagine how much time this kind of development would chew up. The mess that this bastardized process is going to create is going to require some serious consulting bucks to clean up. Good thing your organization has never gotten used to the cycle times of Agile development, or the quality for that matter.

15. Pick the next feature or requirement to work on and repeat these steps

If you get to step 15, and you still find yourself interested in following this abomination of a development process again, please feel free to continue to inflict it upon yourself and repeat these steps on the next feature. Otherwise, just go back to the waterfall method that you're used to and do some end-of-line testing. I hate to recommend that, but I can’t see how anyone will benefit by what Microsoft has laid out in these recommendations

A while back the Patterns & Practices group sent out a survey asking the developer community if they should write about their own experiences with Agile development so as to provide Agile guidance to the community. I responded that under no circumstances should they do so. After having to fight with with MSF for Agile people on the inclusion of mere test-first programming in MSF for Agile, it was painfully clear that real Agile development guidance would never make it past Microsoft's marketing people. The Agile practices would have been watered down until they were more detrimental to software success than the outdated approaches espoused by the guidance proffered through MSF.

I hope the P&P folks can see exactly what I was getting at and desist from any efforts that could ultimately make matters even worse. The best they could do is to work from the inside to have this irresponsible and reckless propaganda stripped from the MSDN portal before it damages the adoptability of TDD in the .NET community, and before it continues to damage Microsoft's and MSDN's reputation as a credible source of practice guidance.

[This entry is cross-posted from my CodeBetter Community blog]

The CodeBetter Community presents a free, one-day Test-Driven Development workshop in Houston on Tuesday, November 29th. The Microsoft Houston offices have made their presentation facilities available for the event and they will also be providing lunch for attendees, as well as a copy of Jim Newkirk's MS Press book, Test-Driven Development in .NET, for each attendee.

The workshop covers the following topics:
- Test-driven development process
- Test-first programming
- Refactoring
- Mock objects
- Inversion of Control (IoC)
- Dependency Injection (DI)
- Design and patterns for testability
- Code metrics

Other prizes and giveaways include:
- 2 licenses of Clover .NET Workstation Edition
- 3 licenses of ReSharper

For more information and registration, visit the CodeBetter Events blog at:
http://codebetter.com/blogs/codebetter.com_events/archive/2005/11/14/134593.aspx

My blog is moving to the CodeBetter Community.  I will continue to cross post here for a time.

Please updated readers to the new feed address:
http://codebetter.com/blogs/scott.bellware/rss.aspx

The browser view is located at:
http://codebetter.com/blogs/scott.bellware/

I’m moving on to be a part of a community focused (mostly) on contemporary software development practices.  I will be joining some friends in the .NET Agile community like Darrell Norten, and Jeremy Miller, as well as all-around .NET community guy Jeffrey Palermo.

Much thanks to GeeksWithBlogs and Jeff Julian for many good times on GWB.

Cool stuff… Google and NASA Plan R&D Relationship

Just downloaded SmartDraw.  I needed to sketch some sequence diagrams to communicate federated identity patterns.

I’ve loathed every single UML tool that I’ve ever used.  It seems none of them are geared toward Agile Modeling or UML as Sketch, which are essentially communication and clarification tasks rather than implementation tasks.

The tools I’ve used in the past have tried to pass themselves off as alternatives to code editors rather than the glorified whiteboards that is their real sweet spot.

Putting together a quick UML sequence diagram in Visio is a complete and utter user-hostile experience.  Visio would force you to create a library of classes before you could drag them on to a diagram design surface.  And once you get a sequence diagram all nice and laid out, you’re practically locked into the original layout, as Visio barely understands UML semantics and how to maintain a cohesive visual representation of a diagram once you begin to morph it.

SmartDraw is the UML sketching tool that I’ve been looking for.  It’s the most streamlined diagram tool I’ve ever used.  Building sequence diagrams with SmartDraw was a perfectly intuitive exercise and the tool essentially let me work as fast as my hands could fly – never once getting in the way of the ideation of the problem that I was seeking to clarify through visual representation.

SmartDraw is UML sweetness – unless of course you’ve been drinking the MDA Cool Aid, at which point you’d be better served by a more sluggish tool that is better suited to fleecing your budget and pandering to your fruitless dream of a code-free software development experience.

Q: Does DLinq perform the query on the server-side, or on the client-side?
A: DLinq translates language integrated queries (LINQ) to SQL.

Q: Is C# 3.0 the version shipping with VS 2005?
A: C# 2.0 is the version that will ship with Visual Studio 2005. C# 3.0 is a future version with no specific delivery date/vehicle.

Q: And will the new 'variables' system in C# 3.0 be a mandatory change? Will we still be able to code in a type-safe environment?
A: If you are referring to the "var" feature, it is for type inference. The code will be just as type safe as with inferred typing as the explicit type. You can always specify the type explicitly as before. It is not a "variant" or "dynamic type" or anything of that sort. It is statically typed - just inferred.

Q: I was wondering if you could comment on the dangers of type inference with regards to code readability. I know that in C# especially, code readability was considered to be very important, and that type inference seems to make it more difficult.
A: Certainly every development teams needs to take code readability into account before using features like type inference. Type inference is best used when it make a solution possible or the code easier to understand.

Q: Will DLinq be supported by other vendors (oracle)? Or is this a SQL server only technology
A: The technology is intended to work across multiple databases. We are looking forward to working with ISVs including database vendors like Oracle for broader support.

Q: I would like to see some detailed, non-trivial examples of Expression Trees. Would you mind directing me to some?
A: There is a non-trivial sample on the LINQ Preview called LogicProgramming. It shows you how to get an Expression tree from a Lambda and manipulate that expression tree to do something useful.

Q: Will the Visual Studio IDE incorporate this and allow type-refernece with intellisense/etc?
A: Yes. We will do work in the C# IDE to make sure all of the new language constructs are understodd deeply by intellisense and othe IDE features. Feel free to send suggestions for what you'd like to see in the IDE for C#3.0 to lukeh@microsoft.com.

Q: Can you please post the URLs to the specs so people can read them?
A: For language specs and other information about C# 3.0, please see: http://msdn.microsoft.com/vcsharp/future/

Q: The Linq preview/C#3.0 works without an update to the clr, just a new compiler. Will this be released before Orcas? or are we still talking about a Post-Vista release for c# 3.0 (2007?)
A: We honestly don't know when LINQ will ship as we're really at the first stage of customer feedback. If you've used the Tech Preview, you'll notice many things are missing, like full compiler support, performance optimization and so forth. We will be releasing another LINQ preview that works with the final release of Visual Studio 2005. More then anything, we want to hear customer feedback on the design and the approach.

Q: Will there be subsequent updates to the LINQ bits that work with the RTM version of .NET 2.0 and Visual Studio 2005?
A: We expect that there will be an RTM version of the LINQ Preview.

Q: I'm interested in DLINQ - in particular whether you see this a complete replacement to OR Mappers or just a quick and dirty query. Is it intended to be useable at the Enterprise level
A: We are starting with simple mapping in DLinq. Depending on the feedback we may add more features based on feedback but we will take a minimalist approach and be careful with additions in V1. Whether it suits specific needs will depend on the situation. It is NOT intended to be a "quick and dirty" thing but nor is it going to have the kitchen sink of mapping.

Q: With LINK, our code will be strongly typed..meaning does IntelliSense check if the data source has certain tables and columns?
A: Yes. C# 3.0 is definitely still strongly typed. To accomplish this, DLinq will create lightweight classes to represent the data in your database. When you do queries against your database, they will return objects of these lightweight class type. This will allow intellisense and other IDE features to behave the same as they have in the past.

Q: spec clearly states implicitly typed local variables. Does that mean that there'll be no support for passing vars around
A: var's are not new data types. They are not variants or types that can be reassigned different values. The keyword 'var' is just a placeholder for the real type. When you use 'var' you are asking the compiler to figure out the type for you from the expression use to initialize it. Parameters to functions are not associated with an initializition expression, so there is no means for the compiler to figure the type out for you.

Q: What's new with concurrency in C# 3.0? I've seen speculation over some mixtures with C?. Any of that speculation true?
A: Concurrency is rapidly becoming a very hot topic, given the fact that soon we all will be running on multi-core boxes. Comega introduced the notion of join patterns to deal with asynchronous concurrency, however, there are many other interesting high-level abstractions for making concurrent programming easier. For example, you should have a look at Software Transactional Memory (Tim Harris and Simon Peyton Jones from MSR Cambridge).

Q: We have linq, dlinq, tsql, clr in sql...lots of ways of doing the same thing...is there a best pattern whitepaper on what to use where?
A: Good question, right now there isn't a best practices team, but we are already working with the Patterns and Practices team on providing guidance based on real-world customer usage, through our Technology Adoption Program (TAP) program.

Q: Any plans to expose code model (AST?) for analisys/manipulation?
A: C# 3.0 Lambda expressions allow you to get at the AST for the expressions.

Q: Is var really type-safe? I can write code that uses methods from a String type; but, if something other than a string type is assigned to my var it (I'm assuming) will generate an exception.
A: Yes, var is type safe. When you define a variable, the initializer will allow us to infer the type. Then onwards it will behave exactly as if you had specified the type. e.g. var x = "Foo"; // string inferred

Q: While I can see the underlying purpose of implicit variables and extension methods, the net effect is to make code more difficult to read. Your're lessening the importance of types and constructors which are familiar language constructs.
A: First of all note that type inference does not imply that you loose strong static typing. For example, if you define var x = "Hello World"; the inferred type of x is string, and you cannot subsequently call x.Age for example.

Q: What are the performance implications of using the LINQ style syntax as opposed to standard methods (IE, foreach over string arrays returned by SELECT blah blah seem to not be optimized as normal, and use the enumerator pattern.)
A: The LINQ syntax compiles down to code that implements the enumerator patterns (as opposed to 'foreach' that consumes it). There will certainly be some small cost over just using a loop, since you'll be making calls to MoveNext() and Current as you iterate. We've looked a bit into optimizations strategies such as combining operators and unrolling them to inline code. None of that's in the preview bits.

Q: if DLinq translates language integrated queries (LINQ) to SQL, how connection string and other stuff are specified?
A: DLinq includes a command line tool called SQLMetal.exe that runs against a specific database and auto-generates classes. One of the classes it generates represents the database itself and it inherits from the DataContext class. The DataContext class is the class that has the connection string,

Northwind db = new Northwind("Data Source=(local);Initial Catalog=Northwind;Integrated Security=True";
Table<Customers> allCustomers = db.GetTable<Customers>();

var result =
from c in allCustomers
where c.ContactTitle.Length == 5
select c.ContactName;

Q: I am assuming that string s = "a" is faster than var s = "a"; are you guys using reflection to infer the type?
A: They are equally fast. The 'var' is resolved to the true type (string) at compile time.

Q: What was the reason behind var a = 10; C# is a strongly typed language...why not make the developer explicitly state what they are initializing?
A: First, its important to note that this is still strongly typed. The compiler figures out what the type is, and will not allow you to later do "a = 'c'", since this would not be type safe. Second, in this case, there isn't really any benefit to using var, so you may as well just type int. However, there are two cases where "var" is very useful. The first is with complicated generic types. For instance, the following is very verbose: "Dictionary<Customer,List<Pair<string,Order>>> d = new Dictionary<Customer,List<Pair<string,Order>>>." Here you don't lose any readability by changing to var, and you probably gain some readability by being more terse. The second place var is useful is with anonymous types. Since you can't name these types, you need to use var when you want to assign values of these anonymous types to local variables.

Q: Why replace standards such as SQL and XPath with a similar but proprietary query language?
A: We are not replacing any standards. The specific query standards like SQL and XPath are designed as separate query languages and will continue to be available for our users in our products. They were specifically not designed to handle language integration and hence often not suitable for that purpose. Each of them targets a specific domain and does not serve a unified view of data. For these two reasons, we needed to create LINQ.

Q: It's not clear to me, if I build a select statement querying the database in C#3.0 with lambda function inside the where condition, the where condition narrowing is sql server side or client side?
A: The where condition executes on the server. The lambda expression can be realized by the compiler as either IL (code) or Expression tress (data). The query expressions translate into explicit calls against your collection object. The DLINQ API has a collection object that implements these 'query operators' as methods that collect expression trees instead of IL. Later, when you ask for the results of the query, DLINQ turns the expression trees into SQL queries.

Q: What would be the recommended way (API?) for building LINQ expressions programmaticaly? Will there be something like LINQ AST exposed?
A: The Espresso sample on the LINQ Preview shows you how to do this.

Q: Linq, looks to be heavliy based off the reflection, are there going to be any preformance issues with it?
A: LINQ is not based on reflection. DLINQ however uses reflection to create objects out of query results and Expression trees encoded with property/field infos to construct SQL queries.

Danielfe [MSFT] (Expert): To the question on why do we need var, I blogged about this today at: http://blogs.msdn.com/danielfe/archive/2005/09/22/472884.aspx, but in my opinion the best benefit is projection, which lets you project a wholly new type. So I can say select c.Name, c.City, c.Phone which creates an anonymous type that has *inferred* string/city/phone structure and inferred data types.

Q: In your example I could write x.CompareTo(...). If that var is retreived via LINQ, what happens if the database schema changed from string to int? x.CompareTo() is no longer valid.
A: I am assuming your question is about DLinq. A schema change from string to int? would require a mapping change as well. When the new mapping metadata is compiled in, the inferred type would change and hence the set of valid expressions would change as well.

Q: When performing an iteration, i use the following pattern Is this bad? Thanks.
int i = 0;
foreach (obj a in b) {
i++;
}
A: This can be a good idea in some cases, though it depends on the type of "b". If it is a type that has an efficient indexer, you are probably better using a for loop that then uses the indexer to grab the elements, since your loop is tied to the index variable. It's really a matter of style, readability and the efficiency of the various methods/properties on the underlying type of "b".

Q: Are you planning to make any release of LINQ so languages other than C# and VB could take it in account?
A: The beauty of LINQ is that it is truly a multi-language thing. Any .NET language can already hook into the framework today by just calling into the LINQ API's.

Query expressions, extensions methods, and the conversion of lambdas to expression trees are syntactic sugar on top of these APIs.

For example, the F# folks are seriously thinking about adding LINQ support as well.

Q: How does DLinq handle differences between string comparisons, in C# == is case-sensitive but SQL Server behavior can be case/kana/accent-(in)sensitive depending on collation settings?
A: We use the server collation setting. In the end, there will likely be a way to override the default.

Q: With the improvements to C# 3.0, do you plan to bring more intricate relationships between C# and ASP.NET?

A: We are always looking for better integrated experience. We still have some work planned for ASP.NET -LINQ integration but it is too early to be specific. We are actively working on databinding as well as embedded C# 3.0 code in ASP.NET.

Q: Would var work with a class that does not support a default constructor?
A: var works if the declaration has an initializer. The initializer cannot be an object or collection initializer, but it certainly can be any "new" expression.

Q: Will C# 3.0 have extension properties?
A: We are currently looking into extension properties. So stay tuned.

Q: "var" makes the code really strange there should be intellisense which tells what that "var" is
A: We will have IntelliSense support for var, both for determing what the type is and then and being able to type the variable "dot" and get an completion list. We haven't determined the best way to show the informaiton yet, though the most natural way would probably be to hover over var and have it tell you the type (this will be interesting for anonymous types too...). All of the bits that were released at PDC have only very minimal work done in IntelliSense, mostly to make it possible to write (remove all the red squiggles you would usually get) as opposed to making it easier

Q: The XLINQ impl in C# seems kind of verbose. I recall that COmega made use of a literal style of XML (like VB 9). Why was that not implemented in C# 3.0?
A: Your real issue is that XLINQ does not have an impl. in C#, not that C#'s impl. is verbose. I'm not sure the final word has been spoken on this yet. ;-)

Q: Why can't "Object" be used instead of "var"?
A: "object" can hold references to many separate types at runtime. The type of "var" is determined at compile time and so it has the benefits of strong typing

Q: Anonymous Types are supported with the context of method or where ever they are created on the fly. How do i make use of them out side of my assembly or application? Simply, how to return Anonymous types from a method?
A: Yes, anonymous types can never escape the context of the method they are created in. This is enforced only in that the type cannot be named, so it can't be assigned into any field/property or passed to any method/constructor. Because they can't even escape the implementation of a method body, they cannot be seen by another assembly or application. In short, the answer is that you cannot return an anonymous type from a method. Instead, you should give the type a name and create an instance of this named type to return instead. We expect to provide refactorings to automate this process somewhat.

Q: Its easy to write lot of complicated joins in SQL..writing them in C#..would it not make code difficult to understand and length??
A: We are still considering more explicit joins for LINQ. However, I don't believe that joins in LINQ are any more lengthy than joins in SQL. Do you have a particularly bad example to share with us?

Q: Why does LINQ use "from" first, instead of "select" ?
A: For one thing, this ordering allows for better Intellisense support. If you were typing "select Name, Age, Position from EmployeeDB", for example, we wouldn't be able to provide completion list support for Name, Age, etc. because we wouldn't know where those fields were coming from. By putting the "from" clause first, we can assist you by looking up the possible field names. Just one example.

Q: When will the C# 3.0 implementation actually be available? I don't want to put a whold bunch of attention into something that I'm not going to be able to use anytime soon. (Especially considering that 2.0 isn't formally released yet.)
A: We are actively working on updating the preview but we do not have announceable release dates for C# 3.0. It is the next major release after VS 2005

Q: May we expect a release of Linq that works with RC0 and/or RTM of .NET 2 in the near term?
A: Yes. We are very actively working on it right now and would like to get it out as soon as we can.

Q: if "DLinq translates language integrated queries to SQL", how can it be used on non-SQL sources of data? I've seen examples that use arrays/collections as a datasource. SQL translation seems overkill in this case.
A: DLinq is specifically for databases. There are different API's per domain, so while DLinq is Linq for databases and lives in System.Data.DLinq, you can use the standard query operators to query arrays/collections and those are in the System.Query, namespace. The interesting part will be as more different types of data stores create LINQ-enabled APIs.

Q: BTW, I sense that you are facing significant developer push-back on type inference and a few of the other features you are adding. As someone who would normally prefer to spend my time coding in dynamically typed languages, C# is getting attractive!
A: Thanks -- but note that type inference via the "var" declarator syntax does not make C# a dynamically typed language. At the level of simple stuff like "var foo = new Foo();" vs "Foo foo = new Foo();", the var syntax is simply a "syntactic sugar" that saves you having to type the rather redundant type declaration. But when you add compiler-generated anonymous types to the language suddenly you really need a way to say "I have a strongly-typed variable here but I do not know the name of the type". Consider for example a LINQ query that returns a tuple of (string, integer) -- it would seem strange to force you to declare the tuple type just so that it could have a name in the declaration.

Q: What are the reasons for not implementing Extension Methods using the new Partial Types syntax, and while still restricting member access, discovery, etc...... it seems like the partial type syntax is more intuitive.
A: You can only use partial types to split up the definition of your classes in your project. Partial types cannot span assemblies or be used to add on to types already declared or sealed. Are you are wondering why extension method syntax does not look more like a partial class declaration?

Q: Is intellisense supposed to be working for Linq "commands" - from, where, group, select. Everything works fine for me, but I don't get intellisense. Just curious.
A: The pre-release doesn't have much in the way of IntelliSense support. It will work in the future, but for the moment the work that we've done is simply to stop it from getting in the way (for example, we made it so that it doesn't report parse errors everywhere). So for the moment the answer is no, but in the future the answer will be yes

Q: Are there any plans to extend the C#2 ability to require a class to have a default constructor, to require it to have a constructor with a specified signature?
A: There are no plans, but that sounds like a good idea.

Q: Is there any plans to add a reverse iteration option to the language?
A: I don't think there are currently any plans to do this. IEnumerable<T> is implicitly ordered, so that reverse iteration is in the general case either inefficient or impossible (the enumerable could be infinite). In many cases you can do something like "foreach(var c in foo.Reverse()) {...}". There is also a snippet in 2005 called forr which expands into a reverse interation over a collection.

Q: C# seemed to be created as a language as a step away from legacy support for non-object-oriented code and to improve type safety. How are are 3.0 additions like extension methods and implicitly typed variabes in keeping with this theme?
A: Implicit typing is about type inference, not about relaxing type safety. It is quite different from dynamic typing in some languages.

Again, with extension methods, there is an additional dimension of flexibility - not a legacy feature. These are innovations tried out in some functional programming language and not in legacy languages. So they are very much forward looking and consistent with the spirit of a relatively recent language.

Q: future version of vb (VB9) include XML literals and more sql keywords in LINQ than C# does, can you explain why C# team decide have no XML literals and havejust few core SQL keywords.
A: Designing a language is a delicate process, once you put something into the language, it is in there forever. Hence you have to be extremely careful what to put into the language and what to put into the API.

Most languages I know that have query comprehension expressions all have diffrenet syntax.

For example, in VB we have choosen to stay as close as possible to SQL syntax because many VB programmers already know SQL. In C# the query expressions are more liberal than what SQL allows, for example you can interleave from and where, and select comes at the end; it is more like XQuery's FLWOR expressions. And languages such as Scala, Python and Haskell have their own different syntax.

Q: Does LINQ allow us to paint our arbitrary data-architectures with whatever, metadata or functions that are necessary to support LINQ-style queries..... such as a query to find the all nodes the Nth shortest hop from a specified node in an arbitrary graph..
A: If I understand your question correctly, you want to LINQ-enable your custom data structures and even add methods that are custom to your data structure like FindSmallestNodeHop(). The answer is yes, we are looking to create documentation on how you can implement the LINQ pattern on a different data structures, but the bottom line is that it's fully extensible. If your object implements IEnumerable, you can use the LINQ query commands like select,from, where etc today.

Q: spec clearly states implicitly typed local variables. Does that mean that there'll be no support for passing vars around. If we don't then var will loose usabilityand the power of true type inference is lost once you cannot write var func(var param)
A: You won't be able to write functions using 'var' as parameters or return types. 'var' is not a type, but a place holder that the compiler will fill in based on the initializer. It may be possible to support var as a method return type, since it would be possible to infer the correct return type, however, it would be much more error prone to allow your method declaration to fluctuate because you changed a line of code in its body.

Q: F#? Whazzat?
A: F# is a cool functional language done by Don Syme in MSR Cambridge (http://research.microsoft.com/projects/ilx/fsharp.aspx). From the F# home page: "Combines the strong typing, scripting and productivity of ML with the efficiency, stability, libraries, cross-language working and tools of .NET".

Q: in what situations would var add to readability?
A: Sometimes the type is painfully clear from the right-hand-side of the statement. For instance "var peopleOrders = new Dictionary<Customer,List<Pair<string,Order>>>()" is very likely more readable than naming the type twice. There are some other cases where var does not add to readbility, and in many of these cases, it is prefferable to use the actual name of the type instead of var.

Q: Jomo: would you compare var to const Object (if I could borrow from C++). If so, why not use const Object?
A: Its not the same. 'const object' means the value can't change at runtime. 'var' means the type can't change and this is determined by the compiler.

Q: Is LINQ works only with MS SQL or any SQL or it provides extentions for MSD SQL 2005
A: Current preview is implemented for MS SQL Server 2000a and 2005. We are exploring various options for broader relational database support including working with other vendors.

Q: Are you planning any release for LINQ so any languages could start using it? I mean, for consuming Func<n> objects and so on, because I work in a language that already has some of these features and we would like to integrate it...
A: LINQ standard query operators are just normal methods. Any language that can use generics can call them. As can any language use DLINQ. The languages themselves will have to choose to implement query expressions or comprehensions or other syntax that makes using LINQ simpler.

Q: What plans do you have involving XML Serialization in C# 3.0?
A: XML serialization is a frameworks, not a language related issue. However, in the context of XLinq (the new XML API) we are looking at providing typed access over XML.

Q: I'd like generic constrinaint new(...) with parameters because classes with default constructors are not good. Is it possible to see the feature in future versions od C#?
A: Possible, yes, but unlikely unless CLR is improved to do it as well. There are a variety of interesting constraint changes that the CLR team are looking into. Unfortunately, these kinds of changes will occur in the future (well after C# 3.0)

Q: Why not add the Design Patterns inside the .NET so we can create MVC, MVP, Singleton etc etc
A: This is something the Patterns and Practices team, http://msdn.microsoft.com/practices/, is actually working on where you will be able to download and have templates for specific design patterns in Visual Studio 2005.

Q: will there be strong debugging support for LINQ?
A: Yes, this is something we're thinking a lot about.

Q: I assume there plans to add List<T> to the classes extended by Linq? I seem to recall having to write my own Where() method.
A: The extension methods that are provided by LINQ extend all IEnumerable<T>'s which includes List<T>. You'll be able to use the Where extesion method on arrays, List<T>s, strings and all kinds of other types.

Q: Will DLINQ support INSERT, UPdates and Paging and Stored Procs
A: DLinq does support insert/update/delete through the SubmitChanges() API. We also do have some support for sprocs for query and insert/update/delete that is discussed in the LINQ Preview documentation. We have some paging support through Take/TakeWhile.

That said, we plan to explore DML (insert/udpate/delete command equivalents) and better paging and sproc support. Please send us any specific feedback you have if you find the current support insufficient.

Q: DLinq has been very interesting to play around with. SQL Profiler helps me get my arms around it. But, what I'm not clear on is architecturally what is the best way to implement it. Certainly you wouldn't want to use type-inference to create DAL objects?
A: I think you are reference to anonymous type constructors, not type inference. You are correct, you would not use anonymous types to represent data access layer (DAL) objects. However, you are allowed to use anonymous types to represent read-only projections of your DAL objects.

Q: The XLINQ impl in C# seems kind of verbose. I recall that COmega made use of a literal style of XML (like VB 9). Why was that not implemented in C# 3.0?
A: I am happy that you like XML literals. We have not yet decided if they will appear in C# as well.

Q: Is the listof new features in the 3.0 spec final or any other things being considered. What about functors?
A: We are still open to new things. Do you mean something like standard ML functors? We have functors with lamdba expressions and delegates.

Q: Are there plans to support indexes on in-memory collections?
A: No concrete plans yet but there are discussions for sure.

Again please send us feedback about top priority scenarios / features - especially from active and expert users like yourself.

Q: LINQ looks exactly like SQL, except the Select is at the end. Why not just put the select at the beginning?
A: Paul Vick discussed exactly this question in his blog today. http://www.panopticoncentral.net/archive/2005/09/21/10553.aspx>

Q: Wouldn't a new keyword for extension classes be wise instead of it just being a "regular" static class? That way the compiler can enforce that all methods in an extension class be extension methods? (i.e. public extensions class MyExtensions { })
A: We thought about that too. Its likely we'll open it up again and reconsider the alternatives. Currently we are having trouble finding a good way to define extension properties.

Q: What are the major differences between the C++ Templates and C# Generics, is it better, same or worst?
A: Unfortunately I have to pick an answer you didn't give - they are different  In fact, they are different enough that managed C++ supports both. Generics are implemented in the runtime, while templates are source expansions (so with Generics you don't have to ship the source to people that want to use your library). Generics start off with the assumption that the only methods that you can call on a type argument are those that are defined on object, and you must explicitly state through constraints additional methods to call; templates, on the other hand, verify that any of the arguments that you've constrainted the templatized type with have the methods that you invoke on them, through expansion, at compile time.

Q: I think Reverse implemented by the language or library could be more efficient than self written wrappers...
A: Maybe. We do actually provide a Reverse extesnion method in the System.Query.dll that is part of LINQ. However, it is not very effiecient, since it is implemented for the general case of IEnumerable<T>. For specific types which support efficient indexers, you could actually implement a more efficient Reverse().

Q: Jono: So var s = "string"; s = "text"; is valid, but var s = "string"; s = 10; isn't?
A: That's right. The second won't compile.

Q: Are there any relations between DSL and new C# features?
A: We think of LINQ as features added to C# to enable mini domain specific languages right in C#.

Q: How about to add a dot-like operator (or dot for Nullable) which gives null for x.a.b.c if any of x, a, b, c is null. I think it is useful/important for 2.0 and especially for 3.0.
A: That one is currently on our design-team short list.

Q: DLINQ seems to be implementing change tracking, but the fields in the database-mapped classes are actually members, not properties. Does this mean that C# intercepts member write access, or does it compare the actual to the original values before writing?
A: First thing - you can use either fields or properties to map.

Second thing, we support both comparison based as well as notification (rather than interception) based change tracking. If you use our code gen (SqlMetal.exe) or implement a similar notification pattern, you can get the optimization benefits of copy on write. Otherwise, DLinq defaults to conventional original vs current value comparison.

Q: There's "virtual visual studio" sessions that let people evaluate Visual Studio. Will there be something like this for C# 3.0?
A: The short answer is "not yet". We'll have virtual machines where you can play with C# 3.0 as it gets closer to shipping, but it is still in a tech preview state. Many customers have asked for more Visual Studio 2005 virtual sessions and we are in the process of adding those.

Q: Where we can download the C# 3.0 stuff most likely its Framework 3.0 and when
A: You can download the LINQ preview code from here, today.
http://msdn.microsoft.com/netframework/future/linq/

The plan right now is to make C# 3.0 LINQ features work without any changes to the 2.0 runtime. We do not anticipate needing to add new instructions to IL, etc, to make this work.

Q: Are there plans for/is there already a way to allow developers to create and add their own domain-specific language (like LINQ) to the C# compiler/IDE?
A: Yes, the C# 3.0 language allow you to create DSLs within C#. See the LogicProgramming sample in the download for an example of this.

Q: It has so far been unclear to me how Extension methods will interact with access specifiers - i.e. can my extension method access private data?
A: No. Extension methods only appear to the caller as instance methods. They are still static methods an cannot access private members of the class they pretend to be declared on.

Q: Will C# 3.0 be made to work with Visual Studio 2005 eventually, or will we have to wait until a new Visual Studio?
A: The current preview works with VS 2005 beta2 and we are working on an updated preview for VS 2005 release version. Final C# 3.0 will utilize new and cool improvements like IDE support in the next release of VS (after VS 2005). C# 3.0 will be a part of the next VS release, not VS 2005.

Q: What are the recommendations to use the "var", cause we can use it almost everywhere (Except for Parameters and Return types as mentioned)
A: You should use "var" when you have to (i.e. when you are using anonymous types) and when the type is clear from the context (i.e. "var x = new List<int>()". In other cases, you should strongly consider not using var for readability. More guidance on this will be provided in the .NET Design Guidelines and elsewhere before C# 3.0 ships.

Q: can LINQ be used with WMI?
A: They should be able to work, assuming WMI classes inherit from IEnumerable. If it doesn't work out of the box today, certainly it is one area that we can enable LINQ support for in the future.

Q: could you please explain more about the "this" keyword used when defining and extension method? why is it needed?
A: The 'this' keyword is what signals to the compiler that the method is an extension method. Without it, the compiler would think the method was a standard static method.

Q: The LINQ Project whitepaper is real light on XLinq details. Is there a similiar expression tree evaluation in XLinq that delegates/maps to XQuery?
A: XLINQ has its own white paper that you can find on MSDN. XLinq is an in memory API like the XML DOM. So far, the standard query operators using IEnumerable<T> have been sufficient for XLinq. We have not planned on extending XLinq to capture expression trees. It is unlikely that performance will be improved this way for XML documents. Now, an XML database on the other hand might. :-)

Q: WRT extension methods, my understanding is that you run into method ambiguity problems if you use the same method (select) on both DLinq and XLing in the same file. Will there be a way to get around this ambiguity?
A: No, this should not be a problem. The Select method used for DLinq actually has a different signature which is used to support Expression Trees. There should not be any ambiguities when using DLinq and XLinq together.

Q: To ErikMeijer. "in the context of XLinq (the new XML API) we are looking at providing typed access over XML". Could you please give some more detail or an example of that? Thanks.
A: Assume that you have an a variable P whose value is the XELement <Person><Name>John Doe</Name><Age>42</Age></Person>. Now instead writing

XElement P = ...create an untyped representation of Person ...;
int age = (int)P.Element("Age");

you would like to write

Person P = ...create a typed representation of Person ...;
int age = P.Age;

with no cast and no strings. We are looking at how we can provide you with a strongly typed experience on top of XLinq, possibly, but not exclusively, starting from an existing XSD.

Q: Will the LINQ bits that ship with the VS 2005 RTM be limited to 32 bit Windows as is the case with the PDC bits or will we have the freedom to compile to Windows 64 bit?
A: To be clear, you can compile managed code and target 64-bit support with Visual Studio 2005. You can then xcopy your exe to a 64-bit server or a 32-bit server without needing to recompile for a specific processor. What won't work is running Visual Studio 2005 the *tool* on certain versions of 64-bit.

Q: Will DLINQ have the ability to materialize collections that derive from EntitySet<T>?
A: The current preview doesn't support that but we will look into it.

Q: If I am enumerating through a collection, what is the recommeded way for removing an item from the collection.. the way I have been doing it is enumerate the collection in reverse and remove the item...
A: Some collections/enumerators are robust in the face of having items removed during enumeration and some are not. Here are a few ideas if you find yourself having to remove items from a collection where the enumerator and collection do not keep each other apprised of changes to the collection:

1) As you've discovered, going backwards sometimes can help, because after all, you're then shortening the portion of the dataset you've already seen, not the stuff left to go. This trick is not guaranteed to work on all collections though.

2) Enumerate the collection and create a new collection of "items to delete later". When you're done enumerating the collection, enumerate the "items to delete later" collection and delete all of them. That way you are not enumerating the collection that you're deleting from. If you're deleting a small number of items, this can work quite well.

Q: What kind of changes if any would we have to make on our datatypes to make them compatible with LINQ?
A: All data type are compatible with LINQ. DLINQ, on the other hand, only supports translation of certain types to the database. If you mean, what would you have to change to make your collection types LINQ compatible, then merely implementing IEnumerable<T> would be enough.  The standard query operators in LINQ work with IEnumerable<T>. So right now at least, you don't need anything special beyond that.

Q: I see that VB9 is offering Dynamic Interfaces... will C# 3.0? How about any other features that allow C# to act like a duck-typed language?
A: Dynamic interfaces in Visual Basic are a way to provide a statically typed view over a late-bound call. Without late binding, dynamic interfaces by themselves do not add much value.

Q: Since LINQ runs on 2.0, are other languages/compilers (IronPython, Monad, Mono CSC, etc) free to implement LINQ prior to .NET 3.0?
A: Yes, I've seen blog entries about very-early support for LINQ in both Monad and Mono.

Q: Implicitly typed variables sound really dangerous outside of LINQ, could a programmer completely avoid "var" when using LINQ?
A: Yes, it's possible to avoid var (though it often is extremely useful) by always defining the structure of your data as a new type. This can make projection more tedious since you'll need to define a named type for the result instead of using an anonymous type.

Q: Is the alternate sql syntax available for extension methods written by us? How is the alternate syntax (select, where from) mapped onto my extension methods? Is this just through the name and type?
A: Extension methods are static methods that appear like instance methods on other types. The SQL like syntax of LINQ (query expression) is translated by the compiler into a series of method calls on your collection class. You can use either normal methods on your collection type or extension methods to implement the methods underlying the query expressiontha the SQL like syntax is translated into.

You can add your own extension methods and take advantage of the translation from the query expression syntax (from ... where ... select ... etc.). But the query expression syntax is language-defined.

Q: Re LogicProgramming: Could this example be pulled out and annotated to explain how and why it works?
A: I tried to put explanatory comments in the sample when I wrote it. I realize its a complex topic and sample. Maybe we could strike up a conversation outside of this chat and hopefully I could answer your questions in detail. Best way to contact me is through my blog (http://blogs.msdn.com/jomo%5Ffisher/) from there I'll get you my email and we can go from there.

Q: Are there any examples of using the new select syntax over XML?
A: Most definitevly! If you go to the LINQ website, you will find many examples. The XLinq API was designed explicitly with this goal in mind. For example, the XLinq overview document at http://msdn.microsoft.com/VBasic/Future/_Toc112834469 gives the following example:

new XElement("contacts",
from c in contacts.Elements("contact"
select new object[] {
new XComment("contact",
new XElement("name", (string)c.Element("name"),
c.Elements("phone",
new XElement("address", c.Element("address")
}
);

Q: I have also heard rumors of multiple return methods ( i.e. public int int MethodName(){ return a,b; } ) in C# 3.0I do not believe these are true, but is it a possibility?
A: You can't do exactly what you are describing there, but you can accomplish essentially the same thing using "yield return". For instance, "IEnumerable<int> MethodName() { yield return 1; yield return 2; }" and then "foreach(int i in MethodName()) { Console.WriteLine(i); }". This allows a method to return a stream of values that are returned "one-at-a-time." so that processing of the first return value happens immediately after it is returned and the second isnt' returned till it is asked for.

Q: It is possible to use SQL Server stored procedures with DLINQ? Can anyone elaborate on "DLINQ and stored procedures" topic?
A: Yes. Please see the DLinq Overview Doc (in the LINQ preview) for examples of sproc usage.

Q: Is there something new to support ASPECT coding ( some kind new Attribute types)?
A: We are open-minded about AOP, in fact Anders and I will attend a workshop on aspects organized by Ralf Laemmel on campus soon. Personally, I feel that AOP is still too young a discpline to be incorporated into a mainstream language; judging from the big stream of conference papers on the topic, it is still too much an open research problem.

Q: Do you plan any way of retro-fitting an interface to a class? e.g. If a library class declares a method Print, and I have an interface IPrintable, I would like to be able to note that the lib class implements the interface so that I can use it in a collec
A: No, we don't have a way of doing this directly in the language. However, you can always create a lightweight wrapper which implements the interface and forwards all calls to an object of the type which has the Print method.

Q: You mentioned databinding enhancements. Would that include generic data controls (eg, GridView<RowType>?
A: Great question. Unfortunately, we are still in the early stages here for me to be concrete. We will look at both WinForms/WPF(Avalon) and ASP.NET binding support for sure but the details are not yet determined.

Q: What I meant by "natural language" is the ability to do "from c in foo.GetThings() where c.State = "NY", select c.State" as opposed to having to revert to method call/lambda syntax "foo.GetThings().Where(c => c.State = "NY".Select(c => c.State)"
A: Currently the plan is to only have the 'natural language' syntax for a certain set of keywords. In order to call other extension methods you would have to use the method call syntax...

Q: "Designing a language is a delicate process, once you put something into the language, it is in there forever". Excellent comment. If D/LINQ falls by the way-side, do the other proposed C# 3.0 additions become hard to justify or moot?
A: An interesting thing is, i don't think you'll see query leaving the language space again. The LINQ additions make sense because they are general and substantiate the query paradigm. Libraries like DLINQ may come and go, however.

Q: A compiler option to accept only C# 1.x, or C# 2.x, or C# 3.x would be nice.
A: The 2.0 compiler has a switch that tells it to conform to the ISO-1 standard, which maps fairly closely to C# 1.0. We'll look at that switch again to see whether it makes sense to extend it for C#3.0.

Q: should functional languages such as F# change implementation to use LINQ? E.g., interoperation between F# and Nemerle is horrible, even though both use the same constructs. Is there any idea to create a unified basis, or any place to post/send ideas?
A: Now that the family of delegate types Func<S>, Func<S,T>, ... is part of the framework, and used throughout the LINQ APIs, I would think it makes sense for languages to represent HO functions using this type.

Q: Is there any intellisense or tool available to write the Lambda Expressions?
A: In the prototype bits distributed on the web and at PDC there is very little IDE support for lambda expressions. By the time C#3.0 is released though, we certainly intend to do a lot of work to make sure that the IDE fully understands lambdas and helps you out with typing them. We have a bunch of ideas for this, but we're always interested in hearing any ideas you have as well.
Q: Matt - My extension methods could include Select( this ...) would using the new Select sntax (not the .Select direct extension call) call my Select extension method?
A: Yes, your method would get called in both cases.

Q: Is there any intention of ever allowing multiple inheritance in C#? Sometimes this would be really handy.
A: No. Many cases can be handled using existing constructs like interfaces. And adding multiple inheritance creates a whole set of problems.

Q: Where I can get all the chat back? Sessinon timeout throw me out
A: Transcript will be posted in the Transcript Archive (http://msdn.microsoft.com/chats/transcripts/default.aspx) in the next 5-10 business days.

Q: I don't think requiring that a class have a specific c'tor if the class has no data members is wise.
A: I understand, however, I doubt we'll introduce new syntax or break existing behavior to enable that.

Q: How will the rollout of LINQ affect relate to datasets and/or typed datasets?
A: hopefully, LINQ will enhance use of datasets and typed datasets. DLINQ may be thought of as an alternative of using datasets, but LINQ itself is goodness for everyone.

Q: Is any good way to do deep copy? If we have large objects its really hard to implement the ICloneable, is there any pattern or way to accomplish it (Other then Serialization Way)
A: The only general way to do this for an arbitrary type is through reflection. However, there are patterns for implementing this on a particular class. Take a look at the .NET Design Guidelines document for information about this.

Q: Any thought about using external xml files to specify database mapping instead of attributes. This would make it easier to change database schemas or database vendors.
A: There is tradeoff involved in using attributes vs external mapping file. We have received feedback from both sides and are reviewing it. I have discussed some of the tradeoffs on my blog at http://blogs.msdn.com/Dinesh.Kulkarni/. Keep the feedback coming. We are still in relatively early stage of design so we can best utilize the feedback we get.

Q: Is there a chance to add aspects to C#, perhaps the CLR in a better way than context binding?
A: As I mentioned in an earlier answer, my feeling is that it is too early for AOP to be integrated into the mainstream. It is still to "vibrant" of a research area.

From Dan Fernandez during the C# 3.0 Language Enhancements Chat:

We will be releasing another LINQ preview that works with the final release of Visual Studio 2005.

Here’s an interesting interview regarding some more innovative thinking on the part of the C# team and how it might impact Visual Basic developers:

http://devnet.developerpipeline.com/documents/s=9856/ddj1126793370067/

The interview is chock full of insightful stuff, but still it’s an article from a VB perspective and there are the obligatory asinine commentaries such as this nugget from the interviewer:

You have this basic LINQ functionality, but then languages like C# and VB can choose how they want to implement it. On the C# side, it seems like you hear a lot about lambda functions and anonymous types, and things that make you feel like you have a doctorate in computer language theory, but I'm guessing that from the VB side, you're not going to force VB developers to go Google "lambda functions" before they can use LINQ effectively.

God forbid Visual Basic users have a little intellectual curiosity regarding the career they’ve chosen for themselves and type “lambda expression” into Google and perhaps gain an insightful perspective on this wacky, wacky world of software development.  When did thinking and learning become so un-American?  Did I miss the memo?

At least we’re finally publicly acceding to the notion that the DataSet and it’s Mr. Hyde incarnation, the Typed DataSet, are perhaps not the best way to implement a solution domain.

All this LINQ stuff makes me very happy, but I’m left with this sense of an impending epidemic of thoughtless domain models reverse-engineered from even less thoughtful legacy relational data models.  Out of the frying pan and into the Easy-Bake oven?

To Whom It May Consern,

In regards to prior nay-saying in regards to the utility, revelence, and viability of object-relational mapping and persistence, I have only one thing to say… Oh, hang on a minute… It occurs to me that someone else has just said it better...  Please direct any further inquiry and/or commentary to the the following page :)

Cheers,
Scott

Just installed the LINQ Tech Preview from the LINQ Project page on MSDN.

Good news for those test-driven developers out there who use unit test to explore uncharted waters… NUnit and the TestDriven.NET test runner add-on for Visual Studio work with the new bits.  This really shouldn’t be too much of a surprise, since the LINQ Tech Preview actually compiles to and runs on .NET 2.0.

As per the Project LINQ Readme (installed with the preview bits), “The installer sets up 4 LINQ project templates (LINQ Windows Application, LINQ Console Application, LINQ WinFX Application and LINQ Library) that you can use to create your own C# 3.0 projects using LINQ.”

I added two projects to a new blank solution to get started with LINQ.  Both are class libraries.  One is a unit test library.  Initially, I made a mistake that is really obvious in retrospect.  I created one LINQ Library project to contain functional code, and one plain-old C# Class Library project to contain unit tests.

The plain-old C# library was not compatible with the LINQ Library.  This would make perfect sense considering that the LINQ Library is compiled with the prototype C# 3.0 compiler rather than the C# 2.0 compiler used by the plain-old C# library project.  A resounding “Duh!“ was heard when it occurred to me what a numbskull move I'd made.

The painfully obvious moral of the story is that a unit test assembly used to test a LINQ assembly should also be a LINQ assembly, or more generally, a client assembly of a LINQ assembly should also be a LINQ assembly.  Even though both projects are compiled to .NET 2.0 IL and executed on the .NET 2.0 runtime, the C# 3.0 prototype compiler obviously has to be in play anywhere that the future syntax is used.

Duh!  I need to restrict playtime to some time before 2:00 AM.

I finally figured out the motivation behind the design of many of the bits of ASP .NET 2.0.  I have come to the conclusion that Scott Goo is taking kick backs from training partners and book authors.  What else could explain so many of the arbitrary changes to ASP .NET 2.0 that deprecates significant areas of our current ASP .NET knowledge?  I suspect that if we looked at his bank transactions we’d see many sweet inbound numbers from big names in publishing and cert prep.

In any case, many thanks to Dino Esposito for clearing up how to do this tried and true thing that we used to do with the DataGrid:
http://weblogs.asp.net/despos/archive/2005/06/30/416783.aspx

I thought this was too funny to keep it to myself…  Here’s the walk-in slide for my TDD workshop in Montreal.

If you don’t get it… don’t worry.  It’s not a joke made for the ethnocentricity of contemporary western society.

:)

I got word on Friday that the Montreal TDD workshop is sold out.  This oughta be a great day – a limted number of attendees (30), and it’s taking places in one of the greatest cities in the world.

If you got your space reserved, see you tomorrow.  If not, well… suckers!!  Just kidding.  Come to the event in the Bay Area.

Test-Driven Development Workshops by Scott Bellware

Jeff Julian, founder of GeeksWithBlogs, has started a new .NET podcast with John Alexander.  The show is called PodcastStudio.net, and you can find it, of course, at www.PodcastStudio.net.

Jeff and John invited me on the show to talk about Test-Driven Development (the One True Way :).  We talked about TDD in terms of tools, patterns, persistent misconceptions, the death of software design in the Microsoft Visual Developer world, unit tests vs. integration tests, code coverage, cyclomatic complexity (the One True Indicator :), defects rates (the One True Metric :), and test coverage (the One True Distraction :).

You can listen to the show at:
http://www.podcaststudio.net/ShowNotes.aspx?ShowID=5

Today is Bill Evjen’s birthday.  Founder of INETA, author, and conference speaker, Bill is an undisclosed age.

Happy birthday, Bill!

I’ll be doing a podcast interview on BlaBla dotNet on NHbernate on Wedndesday, August 17th.

This is a French language internet talkshow out of Montreal hosted by MVP Mario Cardinal, RD Guy Barrette, and Eric Cote.  So for any of y’all who understand French, tune in and check it out.  These guys are some of my favorite people in the biz and I’m looking forward to the conversation.

More info at the BlaBla dotNet site:
http://www.blabladotnet.com/NavLinks/Future.html

What's wrong with Microsoft dev tools marketing folks?  I'll tell ya… They couldn’t cut it in their own coding jobs, so they went into advertising.

It hadn't occurred to them that they're also gonna need to demo the tools that they're coming up with spiffy sound bytes for.  Inevitably, this meant dumbing down the tools to make them easy enough to be used without making the products look as intellectually deficient as the corrupt joker running the dog and pony show.

So what do we get in the end?  Tools that enable bad design by encouraging the dragging and dropping of disparate application architectural concerns into inappropriate application layer components, and messy piles of code-crap that can't be tested except with uber-expensive UI robot tools that only produce brittle test suites that are often left to the ravages of entropy and abandonment.

We're not using dev tools that are stupid because Microsoft dev tools marketing folks think that the developers are stupid, we've got stupid tools because these suckers are so pathetically out of their league in this business that the tools have to be dumb enough for them to pull off an effective demo.

A truly horrible aspect of these macabre circumstances is that these are the folks talking to IT managers and stakeholders.  They are setting the expectations for how we do our jobs by providing and enforcing an illusion of software development that only the most irresponsible and unmotivated Babytalker could concur with.

Marketing people teaching developers how to use tools!  This is like asking frat boys to teach resposible beer consumption!  How many more years do we have to accept this sorry state of affairs as simply "the way it is".  C'mon people, stand the hell up and add your voice to the dissent.  Marketers are phenomenally susceptible to strong messages from customers.  They're counting on YOU to free them from their own iniquities.  You’re in the driver's seat.

If all else fails, maybe the onset of recreational space travel as promised by Virgin Galactic (with research funded by ex-softie Paul Allen :) will ultimately provide an economical way to ship these fools into deep space.

Note: The following post is a rant.  It is a rant about initial experiences with ASP .NET 2.0.  It speaks to the experience of a long-time ASP .NET 1.1 developer finding himself in a strange and stupid world built by creatures from another planet.  It does not necessarily represent ASP .NET 2.0 tools accurately as I have about a week of hands-on time with them.  It does however accurately express my contempt for some of the more lame new features of ASP .NET 2.0.

Warning:  This post make use of the word “asinine” far too many times.  Any readers with a history of allergic reactions to this word should turn back now lest they risk anaphylactic shock.

So…

Am I the only one who thinks that ASP .NET 2.0 web projects are a freakin’ joke?

I just started working full time on production systems with the ASP .NET 2.0 and Visual Studio 2005 beta, and all I can say is… those boys and girls over at the ASP team in Redmond must have been huffing a lot of gas when they came up with some of the stinky ideas that became the ASP .NET 2.0 experience.

This is the number 1 item topping my “The ASP Team Doesn’t Interact with Actual Users” list:

No project file for web projects!

There’s no project file for web projects anymore.  There are project files for the other types of visual studio projects, but not for web projects.  We projects are “special”.  They are “special” in that special way that we use the word “special” to talk about things that are, er…, not quite right, but without having to say that they’re not quite right.  As in, “The Visual Basic petition is ‘special’”.

So apparently, we are all men (and ladies) enough to deal with project files in our class library and setup projects, but we apparently just couldn’t hack it anymore when it came to web projects.

“Dammit Jim, there are five projects in this solution… I can’t handle another project file!  Can’t we get rid of one of these project files?  Arrrrg!!”  Sound spurious to you?  Yeah, me too.

So why did the ASP .NET 2.0 developer experience whiz brains in Redmond make project files go away for web projects?  They answer is really quite simple and really quite enlightening when you finally see it…

Without a project file for web projects, there is no way to register referenced assemblies other than adding the assemblies to the web project’s Bin folder – which is what a web project’s “Add Reference” gizmo does.  This is really quite useless if you happen to be using a source control system.  Without having anything but a collection of files in a folder to specify the references, you’ve gotta put that Bin folder under source control. 

When you have a solution with a web project and a couple of class libraries, new versions of those class library assemblies are copied to the web project's Bin folder each time they are built.  With multiple people on the team building the solution, they will constantly get check in conflicts in the web project’s Bin folder because they are constantly creating new versions of the assemblies in the Bin folder with each build in their own dev environments.

The work around is to not put the web project’s Bin folder under source control and to add a post build event to every project “referenced” by the web project to “push” the complied class libraries to the web project’s Bin folder.  This allows the web project to have the right assembly references and to keep the source control mechanism from having to play a part in the web project references dance.

So you see… this is why they got rid of project files for web projects.  Simple huh?

P.S. Scott Allen dropped me a note to say that this is supposed to be fixed by RTM by having the references stored in the solution file.  If this is true, then the web project's references are stored in the wrong damned place.  If they've gotta write code to store this stuff somewhere, why not just store it in a project file.  It's the same damned thing as storing it in a solution file, except that the solution file shouldn't be the home of information that applies specifically to projects.  What if that web project were to be used in multiple solutions, like when you have different solutions for a web project that don’t include the superset of all the class library projects in a solution so that you get quicker build times when doing Test-Driven Development?