Slow websites infuriate users. Lots of people can visit your web site or use your web application - but you have to be prepared for those visitors, or they won’t come back. Your sites need to be built to withstand the problems success creates.

Building Scalable Web Sites looks at a variety of techniques for creating sites which can keep users cheerful even when there are thousands or millions of them. Flickr.com developer, Cal Henderson, explains how to build sites so that large numbers of visitors can enjoy them. Henderson examines techniques that go beyond sheer speed, exploring how to coordinate developers, support international users, and integrate with other services from email to SOAP to RSS to the APIs exposed by many Ajax-based web applications.

This book uncovers the secrets that you need to know for back-end scaling, architecture and failover so your websites can handle countless requests. You’ll learn how to take the "poor man’s web technologies" - Linux, Apache, MySQL and PHP or other scripting languages - and scale them to compete with established "store bought" enterprise web technologies. Toward the end of the book, you’ll discover techniques for keeping web applications running with event monitoring and long-term statistical tracking for capacity planning.

If you’re about to build your first dynamic website, then Building Scalable Web Sites isn’t for you. But if you’re an advanced developer who’s ready to realize the cost and performance benefits of a comprehensive approach to scalable applications, then let your fingers do the walking through this convenient guide.

Learn the tricks of the trade so you can build and architect applications that scale quickly--without all the high-priced headaches and service-level agreements associated with enterprise app servers and proprietary programming and database products. Culled from the experience of the Flickr.com lead developer, Building Scalable Web Sites offers techniques for creating fast sites that your visitors will find a pleasure to use.

Creating popular sites requires much more than fast hardware with lots of memory and hard drive space. It requires thinking about how to grow over time, how to make the same resources accessible to audiences with different expectations, and how to have a team of developers work on a site without creating new problems for visitors and for each other.

Presenting information to visitors from all over the world

* Integrating email with your web applications

* Planning hardware purchases and hosting options to have as much as you need without breaking your wallet

* Partitioning and distributing databases to support large datasets and simultaneous transactions

* Monitoring your applications to find and clear bottlenecks

* Providing services APIs and using services from other providers to increase your site’s reach and capabilities

Whether you’re starting a small web site with hopes of growing big or you already have a large system that needs maintenance, you’ll find Building Scalable Web Sites to be a library of ideas for making things work.

Slow websites infuriate users. Lots of people can visit your web site or use your web application - but you have to be prepared for those visitors, or they won’t come back. Your sites need to be built to withstand the problems success creates.

Building Scalable Web Sites looks at a variety of techniques for creating sites that can keep users cheerful, even when there are thousands or millions of them. Flickr.com developer, Cal Henderson, explains how to build sites so that large numbers of visitors can enjoy them. Henderson examines techniques that go beyond sheer speed, exploring how to coordinate developers, support international users, and integrate with other services from email to SOAP to RSS to the APIs exposed by many Ajax-based web applications.

This book uncovers the secrets that you need to know for back-end scaling, architecture, and failover so your web sites can handle countless requests. You’ll learn how to take the "poor man’s web technologies", like Linux, Apache, MySQL, PHP, or other scripting languages, and scale them to compete with established "store bought" enterprise web technologies. Toward the end of the book, you’ll discover techniques for keeping web applications running with event monitoring and long-term statistical tracking for capacity planning.

If you’re about to build your first dynamic web site, then Building Scalable Web Sites isn’t for you. But if you’re an advanced developer who’s ready to realize the cost and performance benefits of a comprehensive approach to scalable applications, then let your fingers do the walking through this convenient guide.

Great book on web development, with at least one chapter ALL software developers should read!
When I first started reading this book I had certain expectations about the technical level of the content. I was expecting to have a lot of information about webservers, and load balancers, an d database clusters, and maybe software architecture.


I was pleasantly surprised as it covers all those things and more.

First as I’ve done in several of my reviews let me list the chapter titles.

1. Introduction
2. Web Application Architecture
3. Development Environments
4. i18n, l10n, and Unicode
5. Data Integrity and Security
6. Email
7. Remote Services
8. Bottlenecks
9. Scaling Web Applications
10. Statistics, Monitoring, and Alerting
11. APIs

I would recommend this book to any Web 1.0,2.0,3.0 startup trying to get ready to write their first line of code, well before that even.

Chapter three will be a review to many who read it, assuming they have good software engineering practices. Use revision control, use bug tracking, have a simple and repeatable build. This is really a good chapter which really applies to any kind of software you might write.

A general statement about this book, in numerous places where there are multiple options for tools to use, some free, some which cost real money, the author makes a list of the popular alternatives, gives pros and cons and a ball park for cost.

Chapter four, well if you don’t know anything about internationalization (i18n), localization(l10n) and/or unicode, this chapter will resolve that problem. These efforts can introduce complexity into your system, and this chapter and frankly many place later in the book continue to point out the issues which can come up when dealing with not ascii characters.

Well I could write a chapter about each chapter, but then you wouldn’t buy the book, which you should if you want to know about the topic.

I may even read it a second time.

Upbeat and Informative
This is a practitioner’s book. Very knowledgeable, very hands-on, systematic in an expert’s way, through clearly hard-won experience. Fun and irreverent too. I recommend it highly.

So, what’s my beef? It’s not with the book. Hercules, Atlas, or Odysseus?

Great resource, tells you what you need to know if you are just starting in this field
The book introduces the tools, processes, and high level architectures used in building large websites like Flickr, Youtube, etc. It is short enough to give you the high level framework and send you to explore various other books, software tools, etc to get more depth as needed. I found it very valuable.

useful web developer guide
This book is a very useful guide for the professional web developer whose goal is to produce larger database-driven web sites in a scalable, debuggable way. Topics such as how to handle more web requests than a single web or database server can handle are covered thoroughly, in the usual easy-to-follow style that all O’Reilly books seem to possess.
The author has some good experience with scalable web apps, too, having been part of the development team for the Flickr web site. Think about what it must take to receive, store, and display all the pictures that Flickr has to offer nowadays. Many of the chapters contain some behind-the-scenes descriptions of how Flickr handled the given chapter’s topic, which is very interesting to read. Web application development, really any large-scale web site development, is not simple - there are a lot of things to consider. This book can help you track the major details you should be thinking about for such a project, predict scalability issues that may arise, and design for maximum scalability and flexibility in the future.

Great book for building durable web applications
Despite its small page count this book both covers the basics for building a web application (i.e. how to pick a hosting service) and advanced topics such as cutting edge techniques for scaling out. Great as an introduction to building web applications and as a reference.

Very good indeed
Very good material, specially in the current world where people think that their out-of-the box Java application servers will do the job of serving tons of pages a day.

Building web-scale applications
Given the complexity of addressing ’scalability’, Cal Henderson has done an amazing job of producing 348 information packed pages that will keep you glued to the end. This book is a ground up overview of the construction, security, architecture, monitoring, and yes, even scaling processes. Rarely do I find technology books that are cover-to-cover material, but this one had me asking for more, chapter after chapter. Given the scope, some of the sections are brief, but they give you just enough to kick-start your research and fill in the blanks.

If you’re wondering which tools the big players use, how they scale their databases, how they monitor their servers, or even how they go about their daily life - this book is worth every penny.

Excellent, although written for the LAMP stack
I liked the scope and thoroughness of this book, although I wish it had a little more information about writing web applications for the Windows/IIS/SQL Server/C# approach (WISC?), rather than Linux/Apache/MySQL/PHP (LAMP).

Great chapter about internationalization, by the way. I’ve never seen a better description of UTF.

A place to start and return often
If you are a new web developer this is a survey course in what you need to know. Unfortunately if are a new web developer you probably won’t get 3/4rs of what Henderson has to say. He doesn’t really get into detail with any single topic in the book... leaving it to the reader to find more information later. None the less, no matter where you are on the learning curve, I consider this a vital part of any developer’s library. Buy this tomb, read it in chunks if you’re new and if not reread chapters at a time for inspiration of what you could be doing better.

VERY VERY HIGHLY RECOMMENDED!!
Do you have just a little experience with building dynamic web sites or applications? If you are, then this book is for you. Author Cal Henderson, has done an outstanding job of writing a book that is primarily about web application design.

Henderson, begins by looking at some general software design principles for web applications and how they apply to real world problems. Then, the author gets you quickly up to speed with the issues involved with internationalization and localization, and suggests simple ways to solve them. Next, he deals with the integrity of the data received and stored by your application. He also looks at what receiving e-mail can add to an application, how to implement e-mail receiving semantics, and the various pitfalls that occur along the way. The author then continues by dealing with protocols, formats, and strategies for exchanging data between two or more components in a web application. Then, he looks at techniques for identifying and fixing bottlenecks in your architecture, both before they happen and when they start to bog your systems down. Finally, the author looks at the techniques for scaling each area of your application.

This most excellent book looks at application architecture, development practices, technologies, Unicode, and general infrastructural work. Perhaps more importantly, this book is about the development of web applications.

Best Advanced Web Apps Book. Ever
This book, written by the lead developer of always-popular and it is quite possibly the best book (from a PHP/MySQL web app developer’s perspective) ever written. If all of this knowledge comes direct from Cal Henderson’s head then he’s clearly a *very* clever guy.

Covering everything from basic MVC concepts, bottle-neck analysis, code profiling and coding style through to network design, protocol choice and security, this book is really quite amazing in how much useful information it manages to pack into such a deceptively small book. I found myself highlighting large portions of entire pages and then realized that there wasn’t much point because the whole thing deserved highlighting.

If you’re using PHP and MySQL (in particular) to build web-based applications that might one day serve more than a couple people - you should get this book and read it cover to cover.

Must Have: no two ways about it
I am the CEO of an Internet company out of India. We have several properties, 2 of them ranking in the leading websites of the world. I have always struggled with software professionals who would take too long and raise too many bills for developing our online applications. I did not know how to fight them, I just had to listen to them.

Until I got this book.

This book will guide the well informed management executive what web development is all about. How much documentation is required. What tools are required. What influences software design...so you can fight back.

I then gave it to my head of engineering. He gave me a dirty look. A week later he was touching my feet. The book opened his eyes. I think I am about to see a huge change in our software output soon...

A recommended read if you’re new to production level engineering
My development experience comes from university projects -- where you are often given a framework to work within and issues like security or real world performance are out of scope.

I thought this book provides a very nice intuition behind a number of modules -- abstraction layers, coding practices, keeping a database clean, monitoring a site, etc and connects the dots between them.

If any one particular area is of specific importance to you, you will probably need to get the deep details from another book or more likely a series of online documentation and websites. In my opinion though, this book covers what a physical book should -- enough to turn the lightbulb on in your head and show you some examples.

Addresses all aspects of growing a web-based service
From hardware to the LAMP software cache, from performance profiling to SQL query tuning, from source control to database partitioning the author addresses or touches all topics involved in growing a web service (which happens to be my primary occupation these days). The book is well-structured and the balance between each aspect is fairly good. The coding examples are few enough that their emphasis on MySQL and PHP are not a problem even if you don’t happen to use these technologies.

Recommended to anyone who want to deliver a web service at a professional level.

OK - book
I like this book. Very good - easy language. Intresting to read.

Way too technical for me.
I couldn’t get through it and had to end up sending it to my team of developers. The title is very compelling, but from a creative/management perspective, it’s dense. I wouldn’t recommend this book for someone like me who handles every aspect of Web application development except for the actual coding.

I wish all software developers would read this
Unfortunately, I’m finding that there are still some in the software industry--from "two guys in a garage" to the largest corporation--don’t know, follow, or believe software best practices. Suddenly when something goes wrong (e.g. the wrong version of a file was deployed, changes can’t be rolled back, the application won’t scale), everyone scrambles in an effort to figure out what happened. Oftentimes, if simple software practices were followed, many of these issues would never surface.

This book does a tremendous job identifying many of these best practices, identifies how to easily implement them--in almost any situation, and discusses application scaling techniques. As the book mentions, scalability is made up of three characteristics:
* The application can accommodate an increase in users
* The application can accommodate an increase in data
* The application is maintainable

Like any good book on application scalability, this one begins discussing the tiered architecture that is common in so many modern applications, and is a fundamental step in creating any truly scalable application. This follows into a discussion on source control--another fundamental part of keeping the application maintainable.

The author briefly discusses security issues by touching on cross-site scripting (XSS), SQL injection, and the like. The discussion is well written and thorough for the amount of time spent on the topic.

Finally, the author discusses many of the issues related to deployment of web applications, including system monitoring and alerting. There is also an excellent section on load balancing, techniques to keep databases scalable, and caching. Finally, the author ties the final section together by showing how to take data from a live production environment and use that information to continually improve the application.

This is an excellent read--a must if you are in the business of creating web applications. Whether your applications expect loads of 10 users or a million users, the techniques discussed in this book will make your application perform better and be easier to maintain.

Scalable it is...
Excellent book. Addresses the key aspects of Scalability and answers a lot of the why’s. A must read book

Best to start out on the right foot than to play catch-up later...
This is a book that grew on me the longer I read it (and no, that’s not supposed to be a pun)... Building Scalable Web Sites by Cal Henderson. Let me explain...

Contents: Introduction; Web Application Architecture; Development Environments; i18n, L10n, and Unicode; Data Integrity and Security; Email; Remote Services; Bottlenecks; Scaling Web Applications; Statistics, Monitoring, and Alerting; APIs; Index

Henderson is the engineering manager at Yahoo in charge of Flickr, so right off you can figure he knows a bit about what it takes to scale a web site. From that perspective, what he says is worth paying attention to. The book (for me) started out a little slow, with information on development environments and architecture, stuff that I’ve seen covered in other books in far more detail. I was a little fearful that I was going to get high level "best practices" information with nothing much more. But things started to pick up after that... It felt to me that all the information from that point on really focused on how that concept or practice can make or break a site that is pulling down hundreds of thousands (or millions) of hits a day. Things that work great for 1000 hits can be completely broken at 100000. The chapters on bottlenecks and scaling web applications are excellent discussions of how best to *intelligently* handle performance and growth without just throwing money and hardware at the problem. Well worth reading regardless of how big your site is right now.

He tends to assume/target sites that are built using common open-source technologies, like PHP and MySQL. As such, you’ll probably get much more out of the details if you are already conversant with those. But still, the general concepts apply across the board, and it’s all worth covering. And it’s really best to make some of these design decisions up front, rather than down the road once you’re playing catch-up with your traffic...

Essential Book For Optimal Web Design
’Building Scalable Web Sites: Building, Scaling, and Optimizing the Next Generation of Web Applications’ by Cal Henderson is a fantastic book for examining the important need of designing a web site that is adaptable for the future. At my current place of employment we have had to deal with the issues/problems addressed in this book for years now, and having a guide like this would have probably only made things easier.

With any web-based company and relating product, there will eventually be a time that comes when the current architecture just no longer does the job. When that time arrives, the process of addressing storage/memory/code problems can be very costly, or it can be dealt with at a much earlier point in time. This book won’t do the work for you but it will help point out the problems that can result sooner instead of later.

Chapter Listing

01. Intro
02. Web Application Architecture
03. Development Environments
04. Localization & Unicode
05. Data Integrity & Security
06. Email
07. Remote Services
08. Bottlenecks
09. Scaling Web Applications
10. Statistics, Monitoring, Alerting
11. APIs

Another reviewer is correct in that the gems in this books are chapters 8-10. If you read nothing else in this text, focus on these 3 gems and you will have gotten your money’s worth. For any IT individual or engineer that is in charge or the overall design of your web application/site, read this wonderful book and improve the future of your web presence!!!

***** HIGHLY RECOMMENDED

Good Resource, but a bit wordy
I came to "Building Scalable Web Sites" as our team was expanding, introducing two new developers to the existing pair of us. For that reason alone, I jumped at the opportunity to review this book, and once I realized it was from one of the designers of Flickr, that piqued my interest even more. We’d recently moved from hosting our family albums locally to a Flickr website, and so their background and scalability affected me directly.

The introduction chapter started off quite dry, as the author attempted to introduce "What is a Web Application", a step probably not necessary for this volume’s intended audience.

The second chapter opened right up however. Henderson’s analogy of a trifle to describe Architecture was genius, with your sponge base all the way to your garnish of sprinkles on top. I went racing into the third chapter, exited about the prospects of Source Control ( a problem with our current environment, and one I only see getting worse).

Unfortunately, the book slowed right back down again, dragging through too long segments on Release Management, Issue Management Strategies, and the like. I took longer and longer breaks to come back to this section, almost leaving the book to the side here.

The book continued in this fashion, some bits of great insight and interest, but scattered with wordy, heavy sections that seemed to strangle the pace. As a Higher Education Programmer, Unicode was completely irrelevant to my projects, but the section on Canonical Holes brought me right back in again.

SQL Injections kept me reading right along, but a whole chapter on email in your web application had me drifting again. In summary, Henderson goes into great detail where you need it in some great areas on Scalability, but I’d not read it straight. Find the chapters that relate to your project or your goal, and you’ll find a great resource.

The index is great for this purpose, with well thought out keywords that I’ve already found myself referencing even though I’ve just only finished the book. The lean of the volume is pretty heavily LAMP, with several Linux/Unix only references and software leads, which would be great for some audiences, but in our ColdFusion/IIS environment, I found myself searching for a tool that was described in the book only to realize that it didn’t support Windows Servers.

It also focused heavily on scaling up to millions of users, and I think many system administrators would be more interested in a quicker, dirtier look at taking their dozens to hundreds of users into the thousands and tens of thousands of users instead. Preparing that heavily for growth at these early stages would slow production much too far, in my opinion.

And enjoy the trifle analogy. Mmm...trifle.

An alternative to the J2EE/.NET world view
Finally a book that expresses how scalable sites are developed using open source tools like Perl, PHP or Python. And it not only covers the technical aspects of writing scalable sites, but also some of the process elements like source control and revision tracking.

This is an excellent book. It’s well writtent and treats the reader with respect. It imparts ideas without going through every miniscule detail of implementation. I highly recommend this book.

Get this book if you’re a developer or engineer considering doing a Web startup
Though it’s nothing like a step-by-step guide (and it doesn’t claim to be one), this book appears to be the closest thing available to a nuts and bolts look at managing the technical side of doing a Web-based startup. There’s lots of code inside, but the book isn’t built around a single, extremely contrived, case study like an online wine store. The chapters follow a general pattern: a topic (like bottlenecks in your application and platform, scaling, or monitoring) is addressed and some rules of thumb that describe the way that the author feels things should be done are set forth and explained, with numerous very specific hints and factoids mixed in along the way.

The table of contents doesn’t seem to have been posted here, but you can find it (and a sample chapter, "Data Integrity and Security") by searching for "oreilly catalog scalable web sites" on Google.

Starts out off-topic, and then WHAM...
...it’s all there.

Maybe it’s my background, but I found the first seven chapters to be....dull, and not directly about scalability. To be honest, I almost set the book aside and considered it money NOT well-spent. Then things started to heat up in Chapter 8, and in Chapter 9 it all comes together. That one chapter (9) is the highest density of useful information about website scaling that I’ve ever seen. There are literally gems on every page.

Make no mistake. This book is more of an overview of the landscape, with brief asides that are clearly brain-dumps from his Flickr experience. The author manages to touch on every topic area that matters, and provide simple overviews of the options available and when they should be applied. In that sense it’s more like an informal design patterns book (lots of "yeah, I knew that" and "Ah! I had a feeling there was a pattern there" moments), with just enough detail to let me do intelligent googling for deeper insights on analysis, design, and construction of scalable systems.

Chapters 8, 9, and 10 make the book worth every penny.

Scalability -- A Hot Topic
When I first starting working on Web application development teams, I was a bit overwhelmed by the number of skills and range of knowledge needed to drive the project through establishing the technical foundations; design, development and testing iterations; to final staged release. Lots of things got discussed in team meetings that I had barely a clue about. Not only do I wish I had this book *then,* I wish all members of my teams could have it *now.*

Cal Henderson has a wide background in the area and is lead developer for Flickr, the photo sharing site that has gained deserved popularity and is often mentioned as the quintessential Web 2.0 application.

The author does an excellent job of spreading out before you the whole process at a high enough level so the book can be valuable for managers, designers, and all sorts of people involved in putting out the final product.

His focus is on program design and design implementation issues, not programming as such. Code is not neglected. Many points regarding design implementation are made with code examples and solutions.

I find this book so personally valuable in grounding me in a complex process, I give it the highest Amazon rating even though I found aspects of the book’s organization to be completely incomprehensible. This is a book about scalable Web sites and applications but the author does not define scalability nor does he deal with the broad Web issues (like the scaling myth) until 60% into the book! Chapter 9, titled "Scaling Web Applications" should have been much closer to the beginning since it was a high-level view with no code, as the other chapters. I was also befuddled with placing the chapter on internationalization, localization, and Unicode so early in the book -- even before the chapter on data integrity and security. There is nothing like a mind-numbing Unicode glyphs and grapheme discussion to kill the pacing of a book! Skip the chapter entirely or read it last.

Organization anomalies aside, the author has a good writing style and he does not view humor as a blunt instrument. His four page analogy between layered architecture and an English trifle was worth the space he took. That’s high praise from a guy who does a job that Henderson likens to whipped cream.