C# can be written with any text editor, like Windows Notepad, and then compiled with the C# Command line compiler, csc.exe, which comes with the .NET framework. However, most people prefer to use an IDE (Integrated Development Environment), and Microsoft offers several options for this. Their flagship is Visual Studio, which can be used to work on every possible aspect of the .NET framework. This product is very advanced, and comes in several editions. Visual Studio is not exactly cheap, and might even be too advanced for hobby programmers.
With .NET framework 2.0, Microsoft introduced the so-called Express versions, targeted at hobby programmers and people wanting to try .NET, and they continued this tradition with the later release of .NET 3.0 and 3.5. The Express versions only work for one language, like C# or VB.NET, and miss some of the really advanced features of Visual Studio. However, they are free and will work just fine for learning the languages, which is why we will use it for this tutorial.
For C# programming, you should download the Visual C# Express from http://www.microsoft.com/express/download/. Install it, and you're ready to write your first C# application!