Microsoft's XNA Creator's Club is definitely an amazing reference for anyone with some modest C# skills. While I personally don't care for the video tutorials or the way they package their code samples (everything is downloaded as a zip instead of viewable directly on the web) the quality and diversity is there to get you started. There are also some starter kits up there that make for a nice jumping off point if you're the experimental coding type.
Everything you need to get started is there, too. I'd recommend downloading C# express edition and XNA Game Studio 2.0 for now, but 3.0 should be out shortly. You can get both of these from the creator's club download page.
If you're looking for book-reading, there are 2 books I'd recommend:
For those who are newer to programming or C#, check out Microsoft XNA Game Studio 2.0: Learn Programming Now!. Its relatively short and an easy read, but don't expect it to get into too much detail on advanced game development topics.
If you're more experienced with programming, C#, or even XNA, XNA Game Studio Express: Developing Games for Windows and the Xbox 360 is a great reference. Its a much longer book and the language is a lot more dry and math-based, but both the breadth and depth of the book are greater than the other one. There are a lot of case studies which make it nice, too. Note briefly that this book is based off of GSE 1.0, while 3.0 is just around the corner.
For your first game, you have a lot of options. You could take a starter kit and just mod it out, or do a game from one of the books. Personally, I had a lot of fun taking a simple game form xbla and making it from scratch by myself. The advantage I found with that is that you don't need to spend a lot of time dreaming up game concepts, you can concentrate on just implementation.