Agile adoption of Continuous Integration (CI)

We Agilists talk a lot about the importance of Continuous Integration (CI) as a practice and we talk about it like it assumed; however, when adopting CI in what was a waterfall, iterative or undisciplined environment there will be a lot of hurdles to adoption that have to be planned as part of the CI roll-out. Here are just a few to consider:

  1. Is your code structured in such a way so that different sub-systems can be compiled and tested without the need to compile and test other sub-systems. If is isn’t you will need to refactor your code to remove the dependencies, create proxy classes, interfaces, and interface simulators for the dependent sub-systems.
  2. Is your code organized in the source code repository in such a way that you can check-out entire trees/directories and compile test? If not, you will need to spend time (task stories) to reorganize your code. Be sure to look for opportunities to separate business classes from helper, utility, and other common code.
  3. Are you using a source code repository that will even support the role of a Committer? If not, you will have to spend time migrating to such a source code control (SCC) tool. Why a Committer instead of everyone being able to commit code to the baseline? Simple, Code Committers are responsible for what actually is committed to the baseline. They perform such tasks as code-reviews, architectural compliance, and making sure the code conforms to style and quality standards. Think of it as a nice CMMI kind of practice that improves your product’s quality. You are performing peer reviews, right? What about overall code reviews?

Powered by Qumana

Enhanced by Zemanta

Agile Reading List

This is my Gold Standard reading list for learning and practicing the Scrum Software Development Methodology. Not surprisingly, Scrum can be used for any kind of project, not just software development as it’s Lean Management framework was largely borrowed from Lean Manufacturing principles from Lean Six Sigma, TQM, Kanban, Kaizen and the Theory of Constraints. This list was originally borrowed from the Scrum Alliance’s reading list. In addition to the articles found on the Scrum Alliance’s website, the following books will help you on your journey to Scrum.

All of these books are available via the O’Reilly Safari Books Online website.

Powered by Qumana

Enhanced by Zemanta

Building Scalable Web-Based Applications

Twitter is over capacity
Twitter is over capacity

Scalable web-based applications has got a lot of air-play on social networks like Twitter lately, mostly because Twitter has been overcome by scalability issues and the service unreliable. Having an unreliable internet service is embarrassing and unacceptable. There is no better way to kill your brand image than to have your viral social media strategy fall flat on its face in front of the “instant on” world.

Having said that, I’ve had a lot of requests for guidance on how to build scalable web-based applications that can withstand getting Slashdotted. Building a scalable web-based application can happen incrementally so you don’t have to apply all the principles below all at once. There is an appropriate implementation roadmap that is appropriate for different types of web-based applications. For example, if your site is proving an web-service API (SOAP service) for other web-services or client applications, you should spend time looking at capacity planning for those services with less emphasis on your own public facing web-site. Anyway… here is my brain dump. There is more where this came from.

Application architecture

  • Judicious use of the singleton pattern
  • Judicious Use of the Concurrency pattern
  • Client side form validation
  • Use of AJAX for web-services requests
  • Caching of semi-dynamic data (pre-rendering semi-static pages)
  • Use of MVC pattern
  • Object persistence is separated from the object
  • Use of XML and meta-data instead of traditional row/column SQL commands for each data element
  • Place SOAP interfaces on a separate cluster of servers (aka application servers)

System architecture

  • Use a load-balancer and multiple Web servers
  • Separate your web-application into a minimum of four tiers: UI, Application Services, Object Persistence Layer, Database Services
  • Use a separate NIC on each server and switch for each tier of the application (e.g. web-page server to application server, application server to object persistence server, and object persistence server to database server)
  • Use a load balancer between each tier of of the application (e.g. between web-page servers and application servers, between application servers and object persistence servers, etc.)
  • Use an enterprise service bus object persistence service to make sure object concurrency issues are handled across multiple databases
  • Use clusters of virtualized servers running across multiple physical servers
  • Add performance monitoring services on each virtualized server to check memory, network, harddrive and application utilization
  • Conduct performance tests to determine if any one object needs more server resources, and move those objects onto their own virtualized and physical servers

Network architecture

  • Use lots of partitioning of data across NICS and Physical Switches (aka switch processors)
  • Use firewalls in front of each switch in the top three tiers (UI, Application, and Object Persistence Layers)
  • Place network probes between each firewall and load-balancer to monitor utilization and intrusion detection
  • Place each application stack, a complete web application, application server and database server, in at least two data centers in two different geographic locations using two different internet data service providers
  • Create a private point-to-point network between data centers for transaction load balancing using two different internet date service providers

I know was a lot of technical mumbo-jumbo. Frankly, I haven’t met that many web designers, web-developers, or even a lot of software developers that understand everything I’ve listed. In order to implement the list, it will take more than one skillset from several technology professionals. (Business owners can contact me privately about assessing the maturity of your development staff. You might be surprised.)

If you don’t understand it, I am happy to elaborate privately. Understand that this is what I do when I’m not playing drums in a Jazz combo.