This is an example of the power of social media. I had planned to ignore it but I’ve had enough requests for “official confirmation” that I guess I have to say something :).
Last week someone tweeted “Microsoft phasing out the TFS version control repository and moving to Git. Unofficially confirmed at a TFS presentation”
It caught on and created a minor retweet storm.
It’s simply not accurate. I’ve been told it was the result of a misunderstanding generated in a hallway conversation between one of our MVPs and a customer at a presentation on TFS. Misunderstandings happen – it doesn’t worry me. Social media just allows them to go global quickly :)
As I said when I announced Git support in VS/TFS, our intent is to provide the best centralized version control system and the best distributed version control system. We fully intend to support both, enhance both, etc indefinitely. Both types of version control are useful to different teams and both benefit from the tight integration and enterprise class hosting that we can provide inside TFS and Visual Studio. At the moment, a significant majority of our customers use TFVC. That may or may not change over time. But you can bet that we will continue to work to make both great.
Some will observe that, lately, all our new development work has been in Git features. That’s accurate. That’s simply a reflection of how much further behind Git is, for instance, it’s not supported by code review, CodeLens, My Work, branch visualization and more. We’re working hard to get our Git support up to parity with our TFVC support. In general though, for now and for the future, when we build new features, we try to build them in a way that works well for both Git and TFVC. An example of this is that, as we’ve improved the version control web UI over the past 6 or 9 months, we’ve done it in a way that both the TFVC and Git experiences have gotten better. That’s not an accident – it’s deliberate.
Hopefully this settles the issue and provides an “official statement” that people can point to in order to feel confident where we stand.
You probably didn’t notice that we did our sprint 63 deployment last week. I didn’t blog about it because there wasn’t much to say. We announced a whole bunch of new stuff at Build the week before and there wasn’t much new in sprint 63 beyond consolidating on the Build feature set and preparing for our next event – TechEd in early May. I suspect next sprint’s deployment will also be light as it’s the last deployment before TechEd and most of the work will be preparing for those announcements. I hope to see you at TechEd and, if not, I’ll see you online.
Brian Keller did a very nice blog post on how you can use the new Azure portal to create a super cool devops dashboard: http://blogs.msdn.com/b/briankel/archive/2014/04/10/building-your-dream-devops-dashboard-with-the-new-azure-preview-portal.aspx
Today, at Build, we unveiled a new Azure Portal experience we are building. I want to give you some insights into the work that VS Online team is doing to help with it. I’m not on the Azure team and am no expert on how they’d like to describe to the world, so please take any comments I make here about the new Azure portal as my perspective on it and not necessarily an official one.
Bill Staples first presented to me almost a year ago an idea of creating a new portal experience for Azure designed to be an optimal experience for DevOps. It would provide everything a DevOps team needs to do modern cloud based development. Capabilities to provision dev and test resources, development and collaboration capabilities, build, release and deployment capabilities, application telemetry and management capabilities and more. Pretty quickly it became clear to me that if we could do it, it would be awesome. An incredibly productive and easy way for devs to do soup to nuts app development.
What we demoed today (and made available via http://portal.azure.com”) is the first incarnation of that. My team (the VS Online Team) has worked very hard over the past many months with the Azure team to build the beginnings of the experience we hope to bring to you. It’s very early and it’s nowhere near done but it’s definitely something we’d love to start getting some feedback on.
For now, it’s limited to Azure websites, sql SQL databases and a subset of the VS Online capabilities. If you are a VS Online/TFS user, think of this as a companion to Visual Studio, Visual Studio Online and all of the tools you are used to. When you create a team project in the Azure portal, it’s a VS Online Team Project like any other and is accessible from the Azure portal, the VS Online web UI, Visual Studio, Eclipse and all the other ways your Visual Studio Online assets are available. For now, though, there are a few limitations – which we are working hard to address. We are in the middle of adding Azure Active Directory support to Visual Studio Online and, for a variety of reasons, chose to limit the new portal to only work with VS Online accounts linked to Azure Active Directory.
The best way to ensure this is just to create a new Team Project and a new VS Online account from within the new Azure portal. You will need to be logged in to the Azure portal with an identity known to your Azure Active Directory tenant and to add new users, rather than add them directly in Visual Studio Online, you will add them through Azure Active directory. One of the ramifications of this, for now, is that you can’t use an existing VS Online account in the new portal – you must create a new one. Clearly that’s a big limitation and one we are working hard to remove. We will enable you to link existing VS Online accounts to Active Directory we just don’t have it yet – stay tuned.
I’ll do a very simple tour. You can also watch Brian Keller’s Channel9 video.
Once you go to the Azure portal and get signed up, you’ll see a home screen that looks something like this. In my case, I’ve already created a Team Project called Demo and a website called BharrySample. When you need to create anything in the Azure portal (like your Team Project), just click the plus (+) in the lower left corner.
That Team Project and all its assets, in this case source code, are stored in and available in Visual Studio Online.
We’ve integrated a fair amount of the VS Online functionality directly into the Azure portal, including…
A Team Project home page where you can get an overview of your backlog, source code, builds, etc.
There’s a history browsing experience for viewing changes.
And a source browsing experience that allow you to edit source directly in the browser and commit your changes to the service.
You can easily configure continuous deployment and monitor/explore builds and deployments from within the portal.
You can also do some very basic backlog management.
And last, but certainly not least, you can configure Application Insights for your app and get telemetry directly on your Azure website blade.
As I said earlier, this is just the beginning. We have a ton more work to do to make this a fully realized experience but it’s enough to give you an idea of what we are thinking.
What do you think?
Today is a monumental day for VS Online. It’s the culmination of a long journey from a glimmer of an idea to prototype, private preview, limited preview, public preview, “go-live” and now, finally General Availability. VS Online is ready and open for business.
I’ve written a fair amount about the transition in the service release notes getting posted today so I don’t want to repeat all of that. Read them and if you have questions, please feel free to ask.
I want to take a few minutes to reflect on something else. It’s been really amazing to watch the growth of the service over the past couple of years. You can look at growth in a lot of different ways but one dimension to think about this in is in terms of the hardware behind it. Of course, it’s all hardware in Azure so acquiring it and managing it is pretty easy but it’s still an interesting indication of the growth of the service.
When we first released the private preview, what was then called TFPreview was a very modest footprint of 5 Azure “medium” roles (3 app tiers and 2 job agents). We only had that many for redundancy – they actually ran at less than 10% CPU utilization.
Month after month, the service has grown and grown, in the number of customers, in the depth of usage, in new services, etc.
VS Online is now 212 machines (not counting Application Insights) and most of them are now large or extra large VMs, and are running “warm”. The breakdown looks like this…
- 21 application tiers
- 16 job agents
- 175 build/load test VMs (with the ability to create more when needed)
I left out Application Insights because it would really skew the numbers. It’s a very different service that deals with massive amounts of data and lots of compute over it to digest it down to useful information and metrics. Application Insights alone is 379 machines – mostly small and medium roles today.
I don’t think a week goes by where additional capacity isn’t allocated somewhere in the system. The growth rate is phenomenal and continues to accelerate. Managing a system like this as it continues to grow has provided good lessons in how important automation and good DevOps practices are.
We’ve got a ton of stuff planned for the next several months so stay tuned as we continue to make the service better. As always, we appreciate any feedback you have.
For well over a year now, we’ve been posting release notes for Visual Studio Online roughly every 3 weeks describing the enhancements we deploy each sprint. In our last redesign of the visualstudio.com site, we added a similar news feed for the on-premises products but haven’t updated it very often or provided a ton of detail. Starting with VS 2013 Update 2, we are going to try out publishing VS & TFS release notes on that news feed, providing you one place to go to keep up with the latest capabilities both in Visual Studio and Visual Studio Online.
Check out the VS 2013 Update 2 release notes. You’ll find they are heavy on TFS and lighter on VS. That’s because TFS Update 2 is released and VS Update 2 is a Release Candidate. We’ll post updated VS release notes when it reaches final release.
Hopefully this will be helpful to you in keeping up with all the new stuff coming.
We have also added a link to our Visual Studio Downloads page (it’s at the bottom under Related resources – I’m asking that we improve that) that will give you access to our pre-release downloads. Hopefully this will make it easier for you to find the various Visual Studio related pre-releases that are available for you to try out.
I’m sure we’re missing stuff you’d like to see so don’t be shy about pointing it out and we’ll see about getting it added.
And VS 2013.2 RC released…
OK, this is weird, I admit, so I’ll explain. Today we officially released the final version of TFS 2013 Update 2 and we released a Release Candidate of Visual Studio 2013 Update 2. It’s weird that we RTMed TFS Update to but only RCed VS Update 2. We generally work hard to RTM them at the same time. This time we couldn’t. The reason had to do with some hard dates with regard to VS Online and on-premises TFS compatibility with it.
If you read my blog, then you likely know that we are approaching the end of the Early Adopter program and transition to standard commercial terms. Part of making this transition is giving those customers who choose to transition from the service to on-premises TFS, a reasonably seamless way to do so. In order to do that, customers need to have TFS 2013 Update 2 to enable exporting data from the service. We also want to make sure people have a reasonable amount of time to make the transition. Given all of the dependencies and time window math, we concluded that we needed to have TFS 2013 Update 2 available in final form (we didn’t want to suggest that people export to a pre-release TFS version) by today – hence the TFS release today. That give people between now (April 2nd) and May 7th (the end of the early adopter program) to transition to on prem and avoid paying any service charges for their use.
Visual Studio, however, was not ready to release on the same day so we made the difficult decision to release them at different times. It will be a little confusing for a bit, but not for long. This VS release is a release candidate and isn’t too far from our final release. And, there are no compat issues between TFS Update 2 and any variant of VS 2013 (RTM, Update 1, Update 2 RC). **UPDATE 4/10/2014** It turns out there is one compat issue when VS 2013 RTM or Update 1 is installed on a TFS build agent that has been upgraded to Update 2, hence you should upgrade both TFS and VS on your build agents at the same time. This does not affect what version of VS you can use on the client - only on the build machine itself. You can use this TFS release with any version of VS that supports TFS 2013.
You can download Update 2 here:
- VS 2013 Update 2 RC Patch
- VS Ultimate 2013 Update 2 RC Full install
- VS Premium 2013 Update 2 RC Full install
- VS Pro 2013 Update 2 RC Full install
- VS Express for Windows 2013 Update 2 RC Full install
- Release Management 2013 Update 2 RC
- Agents 2013 Update 2 RC
- VS 2013 Remote Tools Update 2 RC
You’ll notice that VS provides both a patch and a set of “full install” downloads. This update is big enough that we felt enabling people to do a single install (for new installs) rather than having to install RTM and then patch was important.
You’ll also find the KB article here: http://go.microsoft.com/fwlink/?LinkId=390522
Now, let’s talk about what’s cool about Update 2. As usual, I’ll focus on the ALM side of things and let Soma and others focus on the core VS enhancements.
In my post (http://blogs.msdn.com/b/bharry/archive/2014/02/05/vs-tfs-2013-2-update-2-ctp-released.aspx?CommentPosted=true#commentmessage) of the first CTP we released, I covered:
- Work Item Tagging
- Backlog management improvements
- Work item charting improvements
- Export test plan to HTML
- Release Management “Tags”
- An assortment of Git improvements
In my post (http://blogs.msdn.com/b/bharry/archive/2014/03/05/vs-tfs-2013-2-ctp-2-available.aspx) on CTP 2, I also talked about:
- The new “Incoming Changes Indicator” for CodeLens
And new things that I want to point out in the RTM include:
Shared Parameters management
Shared parameters allow you to create sets of test data that can then be referenced by various test cases and shared across the team. This includes traceability from the test case to the references parameter sets and from the parameter sets to the test cases that reference them. When you run manual tests, these parameter values will be in the UI to make them easy to find and you can also bind them in the test context of automated tests to make it super easy to author and execute lots of permutations of data values in automated tests.
Coded UI and Unit test support for Windows Phone 8.1 apps
Not surprising but, with the release of Windows Phone 8.1, we’ve enhanced our testing tools to support testing the new applications you’ll be building.
Cloud Load Testing and Application Insights integration
The single biggest enhancement request we’ve gotten on our cloud load testing service is the ability to capture application performance counters and correlate them to the load test (our on premises load test product has enabled this for a long time). Given that we are building a rich application analytics service in Application Insights, rather than replicate much of that functionality, we’ve chosen to integrate our cloud load testing experience with it. You might wonder why I’m talking about services in an Update 2 post. The reason is that the cloud load testing service is really a companion service to VS Ultimate and part of enabling this integration involved updates to Visual Studio.
Here’s a Channel 9 video on how to configure this: Cloud Load Testing with Application Insights walk-through WMV (2.5 Mbps)
Data Export from VS Online to TFS Update 2
I mentioned it above but I’ll mention it again here. TFS 2013 Update 2 enables data export from VS Online into an on-premises TFS server. You’ll find more details on this in a post I’m doing tomorrow about the service.
TFS 2013.2 introduces support for SQL Server 2014.
The TFS 2013.2 Sharepoint extensions are now unblocked on Windows 2012 R2
Overall, it’s a very nice update with a bunch of new stuff. Of course, I’ve left out a ton – most of the Windows/Windows Phone development enhancements, Azure tooling improvements, web development improvements, .NET improvements, etc. When all of that is taken into account, I think this is the largest “Update” we’ve ever done. I hope you like it and, as always, we appreciate feedback. In the meantime, we’ll keep cranking out new capabilities on the service and start thinking about the next Update.
Wow, this post is late. It’s been a pretty tough month for me personally. Probably the highlight was an ice storm that hit central North Carolina and crushed us. My house was without power for 4 days. The whole area looked like a war zone. There were trees down everywhere. Roads were impassable. It took me 2 days just to clear my driveway so we could get cars in and out. I took off an entire week plus to work on clean up. I’ve logged more hours with a chainsaw in the past month than, I think, most people do in a lifetime. Even now, I’m, at best, half way done clearing debris and repairing fence lines.
So, this post is ridiculously late and my blog has been pretty quiet. Sorry about that. I’ve got a ton of info to get out in the next week or so, so expect a fair number of postings.
A couple of weeks ago, we published the release notes for our sprint 62 deployment. The notable features in this deployment were in our Application Insights service and you can read about them in the release notes.
I want to take a few minutes to discuss one of them, in particular, though. In sprint 62, we introduced the first peek at a new log search service. This is a major addition to our application telemetry capabilities. We describe Application Insights as “Providing a 360 degree view of your application, helping you ensure that it is reliable, performing and delivering the business results you expect.” We’ve got Application Performance Monitoring (APM) capabilities, Usage analytics and now Log analytics. Log analytics allows you to mine all the telemetry (automatically generated or custom logging) to find patterns or to isolate problems. In a live service where you can’t just attach a debugger, logging is a lifeline that enables you to understand what’s actually happening inside the app and analyze the effects of changes that you make.
What you see there today is still just a humble beginning but we will continue to build on it and make it a VERY rich log analysis capability. Check it out when you have a change, give us your feedback and stay tuned for more enhancements.
Last week we released the next CTP for VS 2013.2 and TFS 2013.2 (aka Update 2). Unfortunately, I was unable to find the time to blog about it because I was in Europe speaking at a conference and visiting customers. So, I’m correcting the error now
For TFS, at least, this is a “go-live” release, meaning you can get support and we will enable upgrades from this CTP to the final RTM. In fact this TFS CTP is actually labeled a Release Candidate (RC). It’s a bit weird to have a CTP of VS and an RC of TFS but there are some good reasons for it having to do with the staging of this TFS update, the General Availability of VS Online and the end of the early adopter period that I wrote about here.
As for what’s in this Update, you can read about it in my announcement of CTP 1. The preview of the Git work that I included at the bottom of that post is now included in this CTP/RC.
One other TFS related feature that I wanted to mention is something called “the Incoming Changes Indicator”. It’s a CodeLens indicator and is included in this CTP for the first time. I really like it. It provides information about the code element (method, class, etc) to indicate what changes have been made to it in other branches that have yet to be merged into your branch. It allows you to easily explore those changes to understand the possible impact on any changes you may be making. Here’s a very simple example but you can read a lot more in this blog post.
Part of the reason for designating this TFS CTP as a “go-live” RC is to make sure we get good validation in real-world environments before we finally release it. If possible, I really appreciate you installing it and giving us feedback on any issues you have. We ran an “upgrade weekend” where we helped several customers through the process last weekend to make sure things seem OK and it was pretty successful.
Looking forward to hearing from you!
Today we finished the deployment for sprint 61. You can read the service news to see what’s new. A lot more new code was deployed than is apparent. We’re gearing up for some big announcements in April and May – expect to hear more in that timeframe.
Overall this deployment went reasonably well. Much better than the issues we had late last fall but not quite as well as last sprint. Overall we’re on the right track and I think it will keep getting better and better.
And, if you have an early adoption account, you should see that the extension I described last week is now effective. Here’s mine, as of today:
See you next time,
You can download Team Explorer Everywhere 2013 Update 1 (version 12.0.1) from the Download Center or install the TFS plug-in for Eclipse from directly within your Eclipse IDE (update site URL: http://dl.microsoft.com/eclipse/tfs).
Improvements in Update 1 include…
Build support for Java projects in Git (and more)
The VS Online service blog describes new capabilities we just delivered on the service for building Java projects managed in Git. TEE Update 1 provides the client tools necessary to utilize these new features. Here is a summary of what is now possible on the service when using TEE Update 1:
- Java projects managed in Git can now be built on the hosted build service
- Oracle Java JDK, Ant, and Maven libraries are available out-of-the-box on the hosted build image. This means you no longer need to check these libraries into source control, making it easier to setup new builds. It also reduces the amount of time spent extracting source at build time so your builds are faster Note: this feature applies to builds for both TFVC and Git projects.
- Java projects managed in TF version control can now be built using Maven on the hosted build service (previously only Ant was supported).
The detailed steps for setting up a build for your Java projects can be found on the Build your Eclipse projects page of visualstudio.com.
Our plan is to ship support for building Java projects in Git in the on premises Team Foundation Server in Visual Studio 2013.2. Stay tuned.
Controlling where build output goes
Users can now control where build output goes (for both TFVC and Git builds). This feature has been available to Visual Studio users for a while, and is now available to TEE users via the Build Definition dialog. Users can choose between not copying the output anywhere, copying the output to a UNC path, copying the output to source control (for TFVC), and copying the output to the server. The last option (copy build output to the server) is the recommended option for both TFVC and Git builds and is the default now.
Improving the experience for developers using Git
We continue to improve the experience for Eclipse developers working with Git projects on Visual Studio Online and Team Foundation Server. Two minor, but noteworthy improvements worth mentioning:
1) EGit tools can now be easily accessed from Team Explorer:
2) New options for handling imported Git repositories
Our Git import wizard has been improved to give you the ability to select one or more folders from a cloned repository to import as generic Eclipse projects into the workspace. Previously our import wizard would only allow you to import Eclipse projects discovered in the clone repository. This was fine if you were already working with Eclipse projects in your repository, but if not, you would have to go through additional steps before you could see your files and start working with them. This is no longer a problem.
One other noteworthy addition in Update 1: we have refreshed the language pack for all languages and have added Chinese (Traditional) language support.
Pick up the latest update and give us any feedback you've got. We’ll be busy working on the next one :)
I know this is the second time I am doing this but I believe it is the last. For a refresher for those who don’t remember what the early adopter program is, let me explain. We released the public preview of VS Online in November. At that time, we introduced our business terms, in a preview form – including free access for up to 5 users. You can read more about your options for purchasing VS Online on the Visual Studio Online overview page. We had promised the throngs of people who jumped onto the service early (before we had announced pricing) that we’d ensure a smooth transition for them into the paying service. Part of that transition is an “early adopter program” that enables them to continue to use the service for free for a period while they evaluate the change. Part of our plan has been to provide the ability for customers to migrate their data from the service and to an on-premises TFS with very high fidelity should they choose to do so. Our original expectation was that, this would have been available in the ~January timeframe.
However, readers of my blog will know that we had some post launch reliability issues and, as I described in this post, we’ve had to do quite a bit of work to evolve the service to continue to provide a great customer experience. Since providing a great service is more important than collecting money (OK, both have to happen eventually, it’s just a question of which happens first ), we chose to delay the work on the data migration capability until we had the service in good shape again. In order to honor our promise of an orderly transition for our early adopters, we’ve extended the early adopter program (free, roughly unlimited use). Those customers that created accounts before December 13, 2013 will have the expiration date of their “early adopter (free) status” extended to May 7, 2014. At that time, the early adopter program will end and everyone will transition to “standard terms”. Between now and then, we will enable the data export experience. Stay tuned for a precise date but it’s several weeks away as of this writing.
I know more free usage is not going to be frowned upon too seriously by any of you but, I never like any delay in our plan but sometimes plans have to change. For you early adopters, next week you will see that the banner on your VS Online account will change to show a good bit more time before your account will revert to a standard account. Here’s what mine looks like now.
While I’m at it, let me say a few words about the data migration capability we are enabling.
We promised to provide a smooth transition for early adopters from preview to General Availability (paying) and we are doing so. The ability choose to migrate to an on-premises TFS is an important part of that commitment. We will follow through on that commitment by enabling any customer who decides that they’d rather move to an on-premises TFS (for any reason) to contact customer support and get help migrating their data. This will enable you to export all of your relevant VS Online data (source code, work items, test cases, etc, all with history, dates preserved, etc) and attach it to a TFS 2013 Update 2 server as a new Team Project Collection.
There a few important things to understand:
- Export is not available yet but will be soon (in a few weeks). We will let you know when it is available.
- To migrate your data, you must contact us. You can send email to firstname.lastname@example.org and we will help you. There is no charge for this.
- You can only migrate to a TFS 2013 Update 2 server. If you don’t have that yet, you will need to upgrade. TFS 2013 Update 2 will RTM at the same time we enable data migration. Once you’ve attached the exported collection to your on prem server, it’s just a normal on prem server and can be managed and upgraded to future versions like any other on-prem server.
- There’s no clean, easy, high fidelity way to bring your data back to the service if you change your mind. We don’t yet have an “import” feature. We’d very much like to have one but we don’t yet and until we do, this pretty much a one way street.
- The migration capability will only be available for a limited time – about 6 weeks, to help with people transitioning from the early adopter program. We hope to eventually provide a general purpose data export “feature” but, for reasons I explain below, we aren’t able to do that at this time.
I’d very much like to have a permanent export feature – I think there are lots of scenarios that it would enable, and I haven’t given up on getting there. However, going into the the implementation of this capability we knew it was going to be very expensive. The big problem with it is that the service upgrades every 3 weeks but the on-premises product can’t. That means that when we export, we have to “downgrade” or transform the schema from the then current schema on the service to the schema that was supported in some version (realistically, the most recent) of the on-premises product. That means, for every feature we build, we must not only build the feature and build the upgrade path but we must also build the downgrade path. And building the downgrade path isn’t the most difficult part – it’s validating it on a large enough set of real world customer data to make sure that it works reliably.
At this point, we know we have some large schema changes coming this summer as we enable process customization and other important features people are waiting for. We are not going to be able maintain the the “downgrade” code path through those changes. I don’t like it and I’m sure I’ll get my share of comments reinforcing this but I believe it’s a call we need to make. To manage through this, we have decided to scope the capability, for now, to aiding people through the transition and will consider doing more later. I’m not making any promises but will certainly listen to feedback over the next year.
Of course, this data export/migration feature is not the only way to get your data from the service. It’s just the easiest and highest fidelity way to get it into an on-premises version of TFS. Your data is yours and you can use TFS features and APIs to extract any and all of your data any time you wish.
Stay tuned over the next few months because we are going to have quite a bit of service related news. As soon as TFS 2013 Update 2 and the migration capability are available, I will let you know. I’m telling you now so that you have time to get ready for it in the event you want to use it.
Yesterday, we completed our sprint 60 deployment. With it comes two new features: Exporting test plans as HTML documents and a permission to control who can create new work item tags. You can read more about the new features on the service release notes here: http://www.visualstudio.com/news/2014-feb-10-vso.
As you may know, since ~Oct, we’ve had a run of “bad” deployments that caused unacceptable down time. Due to this we made a number of changes. One was to move our deployments to weekends to reduce the number of people that any unexpected interruptions affect. This was the second deployment to happen over a weekend and thankfully went very smoothly. Sprint 59’s weekend deployment did not go well though and we had a good number of people working all day Saturday and Sunday to get the service back to health. While the weekend deployment had the desired effect – it reduced the impact on customers, it confirmed for us the heavy toll it takes on the team.
In fact, we moved sprint 60’s deployment to this weekend (it was supposed to be last weekend) because we didn’t want to risk having to ask the team to miss the Super Bowl (which, I might add, the Seattle Seahawks dominated). Thankfully this one went smoothly and the number of people who had to work the weekend was comparatively small. However, it’s clear weekend deployments aren’t really going to be sustainable – in the sense of keeping a talented group of people willing to work 7 days a week. Next sprint we are going to try doing the deployment on a weekday evening. I ‘m well aware that no time is a good time for an outage. In fact, I even got some customer emails when 59’s weekend deployment experienced issues, but there are less bad times than others. For context, here’s a graph of activity on our service, starting from Sun morning.
In the end, the most important this is getting to the point where every deployment is a “non-event” (meaning no one even notices it happens because it is so seamless) – both for our customers and our employees. Time shifting the deployment isn’t a fix, it’s a mitigation and while it’s a wise precaution to take, the effect on the team is profound enough that we have to be very careful how much we rely on it.
In that light, we have done a lot of technical work in the past few months to “get back on top” of deployments. It became clear that the scale of the service had outgrown the engineering processes and resiliency we had built in and it was time to do more. Among other things we…
- Doubled down on root cause analysis and post-deployment retrospectives
- Fed all engineering process learning (both from deployment retrospectives and elsewhere) into an engineering system backlog in order to concretely track progress and to help prioritize. From there we farmed work out into feature team backlogs.
- Made significant changes to our pre-deployment testing and scale validation to make it “more like production”. While it will never be fully like production, it is helping catch more issues than before.
- Accelerated work to enable multiple instances for VS Online. In fact, you don’t know it, but we deployed the first “additional instance” a couple of weeks ago. Ultimately, this will allow us to run fractions of our production load on independent clusters that can be upgraded separately. This way we will not have to subject the entire customer base to the upgrade at the same time. The instance we set up a couple of weeks ago, we have affectionately called “ring 0” or (Scale Unit 0 – SU0) and it will be where all production deployments happen first. SU0 is in the Azure San Antonio data center and SU1 (the instance we’ve been using for a few years) is in the Azure Chicago data center. Ring 0 will be an instance where we only put accounts who “volunteer” for the risk of additional volatility – mine will be there . Once deployments are validated on ring 0, they will incrementally roll out across other instances. Eventually we will add the ability for you to choose which instance your account is on and this will give you the ability to locate your account in your region.
- Worked to further decouple services to enable more independent deployment of individual services (TFS, Build, Cloud load, analytics, etc). There’s more work to do here but we are making progress.
- Implemented a new “lock manager” to better enforce some global service policies/best practices that we saw often being validated based on our root cause analysis. I hope to write a whole post on this one at some point and maybe even share the code.
- And much more…
I’d like to think that the smooth deployment this weekend was a reflection of the work we’ve been doing but I’d like to see 3 or 4 smooth ones in a row before I’m willing to give too much credit. We’ll continue to work hard on it. I don’t expect to be at a new “plateau” of service stability and excellent for another few months but we’re definitely on the ascent.
Thanks for your patience and understanding as we continue to work to produce the best developer services on the planet. As always, if you see things we can do better, please let us know.
In the fall, we introduced a preview of a new VS Online service called “Application Insights” that enables you to instrument your app to ensure that it is available, performing and successful. It combines APM capabilities with usage analysis, aimed at developers to help them create the best possible apps.
Yesterday, we released a preview of a new Visual Studio add-in that makes getting started with Application Insights so easy that everyone should do it without even thinking about it. Install the add-in below and get started right away. The add-in will automatically by-pass the invitation code process necessary to get started with App Insights the first time you connect an app.
Download Application Insights Tools for Visual Studio
To get started with a new project, simply create a Web project. In the New Project dialog, make sure that Add Application Insights to Project is checked.
To get started with an existing project, right-click on a Web project in Solution Explorer and choose Add Application Insights Telemetry to Project.
That's it! Then run your Web application locally (or deploy your application), and after 10-15 minutes, telemetry data will automatically start appearing in the Application Insights Portal in the Usage tab.
Additional project types are supported.
Today, fast on the heels of Update 1, we released the first CTP of VS/TFS 2013 Update 2. Although Update 1 was a pretty small release, Update 2 is much larger. This CTP is *not* feature complete and it is *not* “go-live”. It is an opportunity for you to get a peek at some of the things we are planning to release in Update 2. The next CTP (at least of TFS) will be feature complete and “go-live”.
- You can download the CTP here: http://go.microsoft.com/fwlink/?LinkId=390521
- You can read the KB article here: http://go.microsoft.com/fwlink/?LinkId=390522
In this post, I’m going to talk mostly about Application Lifecycle Management (ALM) related improvements coming in Update 2. For the most part, we focused on continuing to “round out” features that have been previously introduced.
Work Item Tagging
In TFS 2012.2, we introduced lightweight work item tagging to easily categorize and find work items. We’ve generally gotten good feedback on the feature but plenty of requests for enhancements. Here’s some things we are adding in this release:
- Query on tags (559 votes on UserVoice) – You can now write and save work item queries containing tag clauses. You simple create a “Tags” contains filter (or does not contain filter). You can combine clauses with “and” and “or” just like any other query clauses. Having it built in like this also means you can create charts filtered by tags (which you couldn’t do before).
- Tags in VS (54 votes on UserVoice) – Previously tags could only be assigned/used within the TFS web UI. With TFS 2013.2 and VS 2013.2, you can now fully use tags within VS. Our Eclipse plugin already supports basic tags on work items and we’ll be looking at adding querying for tags in Eclipse in a future Update.
- Tag permission – Now you can control, with a permission, who can create new tags (if you want to make sure crazy team members don’t pollute your tags with all kinds of random things :)).
- Edit Tags in Excel (30 votes on UserVoice) – Now when you are managing a list of work items in Excel, you can include the tags column, edit it and publish the tags changes back to TFS.
- Object model & REST API support – We added support to access/set tags in the object model and also exposed REST APIs.
We took a bunch of small, but popular requests off our backlog that involved helping you manage your backlog, including:
- Perf improvements – Navigating between the backlog, sprints, etc used to be a post back to the server and was uncomfortably slow. We changed it so it no longer refreshes the whole page and is MUCH faster.
- Non-working days – Starting with Update 2, you can define you’re non-working days (weekends, for most of us) and we’ll exclude them from the burn down calculation so it no longer looks like you are on track when you really aren’t (because you don’t plan to work the weekend :)).
- Configurable start date for cumulative flow diagram – Now you can define what date to start the diagram so that you can spend some time building the plan without the “lack of progress” polluting your chart for the whole project.
We introduced charting in VS 2013 and we’ve done some work in Update 2 to make charts a little more useful.
- Pin charts to project home page – You can now pin a chart to your project homepage so everyone can see it when they visit your project.
- Customize chart colors – Maybe not a huge deal but very nice. Now you can select what colors you want to use in your charts.
Account and Project home page refresh
We’ve refreshed the layouts of the account and project home pages to try to make them even more useful.
Export Test Plan to HTML
You can now export your test plan to HTML for offline reading/sharing, printing, etc. You can choose the level of detail you want to include in the document.
We added a new concept of “Tags” (different than work item tags, just to confuse you) that enables you to tag deployment targets. Tags can be used to treat a set of machines the same. For instance, this allows you to easily deploy the same software to all the machines in a web farm without having to author anything redundantly – just by tagging all the machines the same way.
Git improvements (pre-preview)
These really only get an honorable mention. I include them only because in a previous post, I had mentioned that Git improvements was an area we are focused on. Unfortunately the Git improvements just barely missed the cut-off for Update 2 CTP1 so they aren’t in this CTP but will be in CTP 2. So you won’t find them in this build but expect them soon.
- Annotate – The annotate (aka blame) feature in Visual Studio has been updated to support Git.
- Amend – You can now update your most recent local commit from within VS just like the “git amend” command line allows. You can both go back and fix your commit message and update the files in the commit. Once it has been pushed to TFS, it’s done though.
- Push to multiple remotes – Team Explorer now recognizes multiple Git repos and allow you to select which remote you want to push to/pull from. No more having to drop to the command line for this.
- Revert a commit – You can now easy “rollback” a commit in the event you checked in something that you’ve decided that you really don’t want.
- Progress with cancellation – our long running Git operations now have a good progress experience and have the ability to cancel them if you realize you made a mistake.
Of course, we also fixed quite a few bugs but I’m not going to try to include a full list here.
I’m looking forward to getting a go-live release in your hands so you can give it a go for real but, if you are brave and willing manage through some known issues (see the KB article), check out the CTP and make sure the features work the way you want.
OK, that took much longer than it should have – way too much red tape for this, unfortunately, but the management pack for TFS 2013 is now available. I’ve been getting emails, blog comments and other forms of queries for months asking about it and it’s finally here. You can download it here:
It works with either System Center 2007 or 2012.
You can also get the TFS 2012 management pack if you need it instead.
Today, our sprint 59 deployment to VS Online went live. There’s several new features and improvements that have been highly requested. All of the significant ones are in the general area of work management.
Since we introduced tagging as a lightweight work item “customization” approach, people who’ve tried it have had some consistent requests. One of the top ones was support for querying on tags (we only supported filtering initially). That feature was just rolled out to VSO as part of this deployment – give it a try. Other requests include the ability to work with tags in the VS IDE (rather than just in the web). That feature will be coming in VS 2013 Update 2 (stay tuned for a post on that soon). We’ve got a few other things in the pipe to further complete this scenario and address the feedback so hang in there.
The #2 UserVoice suggestion is:
And this deployment adds the ability to configure your working days (not everyone around the world shares the same days off ).
And more. Read Aaron’s sprint news for more detail.
In our efforts to get better about the instability we’ve been introducing with our deployments, we delayed this deployment a bit and scheduled it for a weekend. The first half of the deployment went well, for the most part, with only one bug that affected a few dozen accounts that had a particularly unusual state of binding their accounts to VSO. We worked around that one quickly and got them working again (to my knowledge only a couple of people noticed this issue). The second half didn’t go so well. We hit a performance issue in the Application Insights pages that caused the site to be unusably slow for a bit. We had to back out that change and get a fix. Today (Wed), we completed the rollout.
I’ve written a fair amount about the issues we had in November and early December. We learned a lot in the process and some of our action items from that have been implemented but not all. Until we finish hardening some more of our rollout process, we’re likely to continue to have hiccups on deployments – the service has simply gotten big enough (both in functionality and adoption) that the earlier processes weren’t enough. I hope, in the next week, to write a post on some of the changes we’ve already done and others that are on the way.
In a previous post on the subject, I explained that this is a pretty small update focused primarily on getting VS 2013 working better with versions of IE prior to IE 10. I also explained that there would be no update to TFS available for Update 1.
You can get Update 1 through numerous channels, including the VS Update facility built into Visual Studio (you will get a notification), from MSDN subscriber downloads or from the Microsoft download center here: http://go.microsoft.com/fwlink/?LinkId=301714
Other useful links include:
KB Article: http://go.microsoft.com/fwlink/?LinkId=331377
Release Management update: http://go.microsoft.com/fwlink/?LinkId=386812
Remote Tools update: http://go.microsoft.com/fwlink/?LinkId=386813
The good news (particularly for TFS users) is that Update 2 is very close behind Update 1 and there will be a new version of TFS released for Update 2. We're just wrapping up feature work on Update 2 this week and will soon be releasing our first CTP. We’re just beginning the process of polishing, bug fixing, etc. to release a solid Update.
In the next couple of weeks I’ll write a post describing the features that we’ve included in TFS 2013.2. Our two biggest areas of investment in this release have been Agile Project Management (and general work tracking) and the next increment of Git tooling. I’ll also cover the highlights of the VS improvements in VS 2013.2 – there are quite a few. Stay tuned for a more complete description.
We have two Great Pyrenees dogs (a male and a female) who guard our goats and keep the coyotes away. When we got them, we decided that we would breed them (they are not related) at least once. Almost 3 weeks ago, Artemis’s litter of puppies was born. It’s been fun to watch. My wife (the vet in the family) happened to walk in just as Artemis was starting so she stayed and watched the whole thing.
The puppies started so small – small guinea pigs and have been growing fast. They sleep a lot. And their eyes just started opening about a week ago. Yesterday, I was out in the barn checking on everything and took a picture that seemed worth sharing. I call it “A Pile of Puppies”. They did it all themselves, without any help from me. There are 6 of them there. Can you spot them all?
I found a nice web article that covers the “puppy process” if you want to know more.
On Thursday, Jan 23rd, Chuck Sterling and Manas Maheshwari will be presenting a Webinar on using VS Ultimate along with VS Online to get started with load testing quickly and easily. You can register to attend here.
I honestly can’t imagine how anyone could consider shipping any web site or service that’s part of some important business function without incorporating load testing into their development regimen. Load testing helps you uncover bugs, deadlocks and race conditions that only happen under load, helps you find performance and scale bottle necks that drag the app down in peak hours and helps you find memory and other resource leaks that can cause the app to degrade over time. It’s a critical part of ensuring the app is always working and that you aren’t fighting fires every day.
For additional info, you can read a post that Ed Glas wrote a couple of months ago about how we use load testing to validate TFS and VS Online.
Hope to see you there,