How Google Tests Software

It is so exciting being part of an industry that is rapidly changing and won't be the same in 5 years or 10 years. Change means opportunity...opportunity not just to "fix" the pervasive misconceptions of what test is and how it should be done, but opportunity to cross into uncharted territory and come up with solutions that no one has ever tried before. That particular point - the idea of the future of test being open and uncharted - is incredibly cool. The book promotes ideas that are radical now but that eventually will become the norm - or they will be tossed as failures and replaced with something better. But the point is that opening up the discussion gives people license to think and experiment, to implement new things that could become great successes or massive failures. Either way, it's a step in the right direction, because test needs a lot of experimentation and empirical thinking right now to go from where it is to where it could be. I enjoyed the practicality of the book - it provided examples of real issues and how they were approached and solved, or not solved. I also enjoyed the lack of dogmatic "this is the only way to do it" mentality. I felt reading it that the authors were simply opening up and sharing, and that if someone was to come up with a great new idea to try, they'd give it a shake. Great pioneers and scientists care only about progress in their field, not about who gets the glory for the progress, and I felt like that's what this book was after - progress. Great book to get you thinking and learning, and where there are gaps or open-ended questions, this is the territory for opportunity.

I learned to develop software in the 1990s and started full-time work in the 2000s. I took time off to study other fields and returned to the practice in 2012, about the time this book came out. In the last 13-or-so years, I’ve noticed that the art of testing software has changed significantly. Twenty-five years ago, I started to code in an academic lab where we did our own testing out of necessity. In industry, I found teams arranged to perform tests. Now, it seems that testing has become a developer-only task again. I didn’t understand the backstory and am always interested in improving my skills. Thus, I purchased a few books on testing in general because I couldn’t find many newer works on testing that transformed the field. Most of the books, like this one, haven’t added a ton of new content since the 2010s unless it is related to security or automation. Even though this book is 13 years old – an eternity in software development – it marked a historical culture change in software development. It taught that developers should be intimately involved in testing of their code and that testing code is not an intellectually inferior task. It sought to reframe the development-test cycle back into bettering the product. Indeed, the conclusion sought to get rid of the role of a pure test engineer in favor of a hybrid developer/tester approach. This book focuses on managerial principles that implement this transition. It doesn’t address the nitty-gritty of testing practices and instead expounds upon how Google rearranged testing as a part of every developer’s workflow. It used to be that “test” and “dev” would cycle back and forth many times to fix bugs. Google’s then-new approach united the process and enhanced both speed of delivery and the quality of the product. This approach suited the world wide web better. Few organizations work under the older paradigm anymore, and this book ushered in the change. The older paradigm helped develop software running on separated devices, but the Internet facilitated continual deployment where fixes could be shipped quickly with direct user feedback instead of lengthy testing runs. In turn, many former testers reoriented their efforts towards finding security holes and other software-related tasks. It’s interesting to trace this history and understand the drivers behind the culture change. I’m not sure understanding these nuances are necessary for most developers and especially not so for newer developers, but they enrich my knowledge of how coding practices have evolved in my lifetime.

It's all about expectations. I didn't expected understand all details of testing inside Google by reading this book. First because it would never fit into 300 pages, second because it is obvious that they have many "secrets" and innovation that they are not able to share yet. I actually expected see a little about how they see testing, what they think is important in testing nowadays and in the future, what is the problems they had and, in a high level way, how they overcome (or not) those problems. And I am happy with what i read. This book made me realize they are innovating a lot, that they do a lot of amazing work there (and they share it, what is equal important), but at the same time they are really close to us in many other aspects, such as thinking about how make builds more reliable and fast, how deliver high value software as fast as possible, how to bring tests from a end-to-end level to a service level, how take the most of cloud infrastructure to benefit testing and so on. It is not a book for testers, it is a book for the ones who care about testing and mainly, the ones who care about deliver the right software in the right way.

The main contribution of this book, besides being an excellent read for anyone who considers working at Google, is the proclamation of how seriously software quality should be taken. Paradoxically, the book is technically complex, and yet those who should really read this book are managers - who often have a factory view of software development and fail to understand that high quality costs less. This is such an important lesson that needs to be learned by the software industry, that the effort of any author to demonstrate this point must be lauded. Unfortunately, the book has two main drawbacks: one is that it is so specific, that it is unlikely to be of much help to other companies. The testing framework Google has built is extraordinary, but it is not a framework that can be easily reused in other contexts: it is highly web-oriented, and it leverages Google's distributed infrastructure. The other is that the book is highly romanticized. It almost reads like a romance, and SETs are the heros. On one page, it is described how a developer can launch hundreds of tests and get coverage reports with a one-line command, a hallmark of efficiency - but on another page, a code sample using the testing framework is presented and it consists of 90% boilerplate code. The book is riddled with confrontations between the idealistic reality the authors describe and how that vision falls short of reality, be it in code samples or interviews with Googlers. Also, SETs are presented as superhumans - in the section where the hiring requirements for a SET are listed, one learns that in order to be a SET at Google, one needs to be a genius. Not a Google-employee level of genius, but an Einstein-who-can-also-read-other-people's-minds level of genius. They are supposed to be able to code any feature and any type of test, while at the same time never losing sight of the big picture even when writing the lowliest code. They are supposed to have the broadest view of the systems among all engineers, while at the same time not even being full-time on the projects! Clearly, if such people existed as to be able to meet the SET requirements that are listed here, Google would never have had the need to address software quality issues in the first place!

I saw James Whittaker speak at STAR West in 2011, and he gave a keynote titled "Test Is Dead". His talk was essentially a teaser for How Google Tests Software that he co-wrote with Jason Arbon and Jeff Carollo. The premise of the book is that testers need to have engineering skills (sometimes to an equal extent as software engineers) in order for the testing discipline to reach first class citizenship on equal footing with development. The argument is aligns well with the movement toward agile software development methods. The book goes on to breakdown testing responsibilities for software engineers (SWEs), software engineers in a test role (SETs), and Test Engineers (TEs). Almost half of the book deals with the roles and responsibilities of the TE, and in the Google model, they do have a higher-level role in testing. In essence, it breaks down like this: * SWEs write unit tests for the software they write * SETs write tools to enable testing without external dependencies and write automated functional tests * TEs coordinate the overall testing activities for a product and focus on the user by doing exploratory testing In addition, the book also outlines a number of tools (many of which have been open-sourced) that Google uses for testing in the context of these roles. The majority of the content focuses on web applications (it's Google after all), and some of the ideas won't apply if the majority of your development is for internal customers to your company - since you probably have user training and rules about frequency of release. However, I would say that you could apply 80% of the ideas in any context and probably adapt at least 10% (if not more) of the others to your situation. Also, there is also a chapter on test managers and directors that has interviews with a number of prominent Googlers. Then, the book ends with a discussion on the future of the SET and TE roles at Google along with some of the errors Google made. Google embarked on the transformation in 2007, and my company is currently trying to do something similar. I hope to be able to leverage these ideas in the months ahead. I recommend it to anyone who is or expects to be involved in such a change. I would also recommend it to any tester in an agile development shop. You may not agree with everything in the book, but tells of the future (if not the present) for much of the software testing industry.

This book is is a rare peak into the development and testing culture at one of the largest software producers today. Filled with insider commentary and interviews, the book presents Google's approach to integrating testing into development. Case studies such as the Chrome browser, Chrome OS, Android show that there isn't a one true approach to quality, and show how and why people at Google made tradeoffs between automation, exploratory testing, pushing testing into development, informal checks and formal plans. I found the discussion on scaling testing and integrating developers and testers particularly interesting as it offers a lot of good advice on how the roles of developers, testers and test managers changed at Google. If you've recognised your organisation in my presentations at StarEast and Oredev (Sleeping with the enemy) or Eurostar (Death to the testing phase), you will surely find a gold-mine of great advice on how to better engage testers and developers into a whole team approach. My top three picks for ideas that that should inspire you are: Test Certified program that got entire teams interested in building quality in Awarding people readability levels for important technologies and relating that to roles they play in code reviews and commits, effectively building something as effective as opensource committer levels to scale out development to a large number of people Software engineer in test role and career path discussions Developers, testers and managers working for larger organisations will surely find the ideas on managing test plans interesting, in particular the ACC Matrix. If for nothing else, you should read this book and keep it close because it sheds light on this innovative practice. I first heard about this technique two years ago at a conference, and since then I've been fortunate enough to work with several companies that let me try it out, with fantastic results. I think ACC is one of the most promising value/risk based test management tools today, and it has helped several of my clients re-evaluate how and what they test and reduce their investment in testing by 80-90% while getting a better return on the time spent. So far, there's been very little material published on how Google actually uses the technique, and this book will provide a valuable reference for many people out there trying to implement it. It's a bit unfortunate that the level of technical detail varies greatly, so I'm afraid that low-level technical stuff might put off managers from reading the more important high-level aspects on scaling. I give this book 4 out of 5.

I really enjoyed this book. I became a test manager three months prior to reading it. I wanted to see how a large company ran testing. I was really impressed by the responsibility that Google places on its software engineers for quality. Testing and quality belong to the entire team and not just a few people with the word Quality in their job title. The end of the book was a surprise. It made me envision a future where quality was just built into the framework for developing applications and not something bolted on at the end. The tools are getting better and the practices are improving to the level that testers can focus on the user experience and not basic correctness. The hope for testers is that they will be more of a user advocate than a button masher.

I ordered this book because it was a Jolt award finalist for 2012 and the intro/look-inside sounded interesting as well. However, after reading it (cover to cover except appendix a/b) I am contemplating the reason behind its inclusion as Jolt finalist. The content might be relevant to some readers but its general applicability is dubious. There ins't much that i would be able to use from what i learned from this book. I wouldn't recommend this book to anybody except if they want to know job roles at google in testing domain.