I have been doing a lot of research into distributed platforms/applications lately, as I am building an application that will need to take advantage of a distributed platform. While I haven't been able to put all of the below into practice simultaneously, I have tried each individually (as well as many others) to help decide what should (or shouldn't) be in my application.
In my opinion, the only way to be continuously successful with any distributed application is to implement the following:
- <abbr title="Hierarchical Model–View–Controller">HMVC</abbr>, <abbr title="Presentation–Abstraction–Control">PAC</abbr>, or similar system architecture
- Package management/build tools (e.g. NPM, Composer, Gulp, Grunt, etc.)
- Great server & client side caching
- <abbr title="Testing Driven Development">TDD</abbr> + Continuous Integration service
- Easily/quickly scalable servers
- Good in-code documentation
- Use plenty of open source libraries
Without these, managing and extending a distributed application seems to just become too cumbersome. Beyond these things, there are plenty of "optional" extras (such as using a build language, semantic versioning, and/or Git) that will greatly help the process.
Also, while it may take ~1 second to send a page to the user from the time the request was sent, the machine will usually process the request and move on to the next after about 200 or 300 milliseconds; the rest of the time is spent jumping between servers/routers to get from the server to the user.
Of course for more complex applications, the process time can increase some. But if you are using a using a distributed application, this increase can usually be negated or at least minimized.