ingo() -> tech. % Ingo Schramm

The last years we underwent the amazing growth of web communities up to previously unimaginable sizes. Literally tenth of million of people collaborate in a single web community or social network and endow their personal content to give the cloud its worth. The success of the large communities is followed by a large number of copycats all over the world trying to participate in the business model. This is what I call the goldrush of the communities.

It never was actually a goldrush but simply a rush. The gold is still missing. Only a handfull of these communities earns profit. Most of them promise profit for the near future and proof by the pure number of members. But the business model to get revenue by advertising only is somewhat controversial. Payed services exist in the minority. People participate on the Web 2.0 for free. Most communities live on the hope to exploit the huge numbers they deal with every day "soon".

For now, there seems to be no place for new communities. The claims are made. The market will consolidate in the next few years and only the largest networks will survive, maybe some niche communities also. If you think about new business you should not try out another social network.

This doesn't mean there is no place for new business in the network domain. Since the huge social networks become more and more platforms for services rather than services by itself the room is open to establish exactly that: services for communities. If you know anything special and how to do it in the internet do it embedable, prepare your service for high traffic and write apps and gadgets for the major networking platforms.

The areas of interest are mainly:

- mobile devices
- location based services
- micro shops
- embeddable games
- multimedia delivery

The goldrush today is not for communities but for services to support communities with valuable features.

Programmers love to talk about performance but only a few talk about scalability, despite of the fact that it is only scalability that counts. Scalability is about how your effort has to grow with your success. This is what you must have in mind if you start a business for the web.

It is always a good idea to plan a new project for scalability from the first steps on. There are some easy task with just a little overhead in development but great benefit if your project reaches the level of success you've intended.

- divide and conquer in size and time
- use a good abstraction layer for data access to be free if you need to partition data
- avoid complicated joins and excessive normalization, they'll kill you if you have to distribute data over multiple data bases and mashines
- do you really need relational database schemas for everything or may a simple key-value-storage work as well
- cache data access from the beginning
- cache more, compute less
- use functional decomposition, partition your system in tiny and efficient units and plug them together by abstraction
- use asynchronous strategies to manage load peaks
- split static and dynamic content carefully, soon you may need a CDN to deliver static content
- apply a good deployment strategy with rollback
- measure and monitor performance and scalability systematically
- scale your revenue in parallel with your technology

Don't miss the last point. It's the most important.

There exist a reasonable number of myths about PHP. One I will challenge today. If you ask an average PHP programmer whether a static function call or a function call on an instance performs better he or she will problably say - without thinking - static. This is wrong.

Sure, if you want to call a method on an instance you first have to instantiate that instance. This takes some time by its own. The sum of CPU time for instantiation and a single method call is in fact greater than the CPU time needed to call a static function. So, if you ask for the time of single calls you will support the myth. But the question is wrong.

In real life you will never do isolated calls. The question of interest is not how a single call performs but how the different kinds of method calls scale. If you measure this you will get a different picture. The instance calls scale definitely better. If you even have just a handful of static method calls on a class the same calls on an instance - a singleton for example - will save you time.

The image below shows a measurement with empty methods and light weight classes on PHP 5.2.5. The point where the two lines are crossing may vary with the complexity of your class.

You've written a not so trivial PHP application and your web site is visited more and more frequently, 1 time per second, 10 times, 20 times per second, more. You transfered your database to a second mashine but still your webserver performance breaks down at 25 request per second. You install a byte code cache like the APC or XCache but your servers still do not deliver more than 30 or 40 requests. What now?

You may spend some weeks to refactor your whole software. Or you invest 30 minutes to fix your file inclusions and you win some 100% performance - even more.

The recipe is easy - watch your server process with strace, take care on file stats and try to reduce it to zero:

- reduce your include_path to a single entry or don't use it at all
- include or require absolute paths
- ensure that each try to include a file is a hit
- avoid file_exists(), is_file(), is_dir(), is_link() etc.
- avoid autoloading

That's it.

Each file system access forces the operating system to switch context and wastes a huge number of CPU cycles. If you have a long include path and the file to include is found at the end you have a lot of useless file stats. Even the byte code cache cannot protect you from this since it stores only files found, not files that do not exist. For each non existing file all the stats and lookups take place again and again. This is what your CPU works for, not your business logic.

Stats are evil.

Hi,

today I open my weblog and you are all welcome to read and comment the posts. I hope to share some valuable experience how to write and run web applications. I am programming in different computer languages for about 20 years and the LAMP stack since the late 90s. The "P" stands for Perl first and since 2001 for PHP. Since then I was looking for alternatives to the somehow overpragmatic PHP technology. On the other hand, the market forced me to use rapid technologies. I tried out JAVA and C#, but both seem not to be rapid enough if you program the Web X.0 for business. Today we have better alternatives in Ruby (on Rails) or Groovy (on Grails). With the raise of the Web 2.0 I am facing server loads I never dreamed of, literally many 1000s of PHP hits - per second. This forced me to adopt strategies of load balancing and partitioning. Now I really know about "divide and conquer". The Telcos deal with high loads for years and so they have tools like Erlang to serve them. I gave it a look and it looked strange and beauty. Not as beauty as Haskell, isn't it? Well, whatever you will read here, it is backed up by practice. It's definitely up to you if you believe me or not.

Kind regards,
Ingo