Web Development
Recent project: PDS Trader PDF Print E-mail
News - Web Development
Written by Tim Black   
Thursday, 07 November 2019 12:15

The majority of my web development work for the last 2 1/2 years went into maintaining and developing new features for PDS Trader, a platform for finding and planning the best option trades.  PDS Trader is made by Quantum Trading Technologies (QTT).  At the center of PDS Trader is its Portfolio Builder interface, where you can layer option trades ("legs") to make their net profit be positive regardless of whether the underlying market moves up or down.  The profit and loss of each leg is plotted on a chart (made with Highcharts), and their net profit or loss is displayed as one line on the chart.  By default, only that net profit line displays.

In the video below, QTT owner Ryan Jones demonstrates the central Portfolio Builder interface on which I worked.


Last Updated on Thursday, 07 November 2019 12:27
Interview questions, and answers! PDF Print E-mail
News - Web Development
Written by Tim Black   
Tuesday, 12 March 2019 22:47

I'm looking for a new job as a full-stack web developer, so thought it might be useful to me and my potential interviewers to share the answers I wrote recently to some questions I was asked in an interview.  Happy reading!

What is important to you in a career?

I want to use my gifts to serve other people and provide for my family, and do the best job I can at that.

Have you ever worked remotely? If so, what challenges did you face and how did you overcome them?

Yes; nearly all of my 19 years of web development work has been done remotely. One challenge I faced was that a client asked for way more work than I could perform, and when I could not complete all of that work, he refused to pay for the work which I had actually completed! I had to let that client go. I learned a lot from that, but especially to communicate with a client early, often, and specifically about the client's needs and the project's progress. In my current job, we use Slack all day, occasional phone calls and screen sharing, Trello for issue tracking, and (rare!) lunches at a restaurant, and this regular communication helps us get things done. Most of the time I'm free to work without distraction, and we have open lines of communication to get help when we need it. For me, this arrangement has been ideal.

Describe the problem solving methodology you would use if asked to implement a new feature in an existing codebase.

Here's how it works in my current job. Typically I will get specific instructions from the product designer describing the new feature, sometimes with screenshots of how it should look. When something isn't described clearly enough or it doesn't quite make sense, I work with the designer to nail down the specifics so I'm sure I've got it right. If the feature is really large in scope, I document the feature's design first, starting with the intended use case, then the UI components & functionality, then the data model or database schema, then any new modules of code needed, then any general API surfaces that are needed. All documentation is versioned with the code.

Then I move to implementing the code. I start by making a new feature branch using git flow. I add the feature branch's name to the Trello card. If I don't already know where the code needs to be modified, to find the right location at which to begin modifying the code, I often start from the user's first point of interaction with the app, so I inspect the HTML source and trace the execution and the data back through events, variables, HTTP requests, functions, back to the database, until I find the location where the new feature can be implemented. I often then use a debugger to step through the execution to be sure of its flow and to know exactly what data is present. In the places where we are able to use unit tests, I write a unit test first. Then I outline the code changes I think need to be made in comments, then I write the code (including docblocks for methods), run it, see if the unit tests pass and the user interaction works right, and debug as necessary.

After implementing the code, I briefly document the changes I made in a git commit message (formatted for conventional-changelog), include the Trello card's URL in the commit message, rebase the feature branch on the develop branch, squash the feature branch if appropriate, and merge it to the develop branch. After testing the new feature and getting approval from the product designer, I merge the changes from the develop to the master branch and push them to production. Then I record any useful notes in the related Trello card (noting the relevant git commit hash; I like how Github automates that part) and move it to the "Deployed" list. That way, if we discover later that the problem isn't really solved, we have a paper trail to follow to fix it better in the future. In a hobby project I used TravisCI to push changes to master & production only when their unit tests passed, to implement a CI/CD deployment pipeline.

How have the SOLID principles influenced your code?

Most of my code has not used complex class hierarchies, and most of my time has been spent writing code that depends on base classes provided in frameworks and libraries, rather than implementing the base classes myself. So I haven't focused much on the whole domain where SOLID principles matter the most. But I have learned aspects of SOLID principles and practiced them as I've matured as a developer. That learning started when I read about extreme programming on the C2 wiki and elsewhere in the early 2000's. While I care to use many best practices (e.g., I aim to write clean, well-organized, maintainable, well-documented and self-documenting code), I also remain a pragmatic programmer, rather than a purist.

Single responsibility principle: I have followed the single responsibility principle more since I learned to write functions so that they can be more easily unit tested (functions that do only one thing are easier to test), and because "don't repeat yourself" is good advice most of the time.

Open/closed principle: I've extended base classes plenty in Backbone, TurboGears, Django, and CodeIgniter, and then overwritten the default methods they provide. Ever since learning how Prototype, Mootools, and other early JavaScript libraries modified global prototypes and so made themselves incompatible with each other, I've avoided monkey-patching base classes--who knows what havoc monkey-patching them would cause somewhere else in the application!

Liskov substitution principle: When I've extended a base class and overwritten the default methods it provides, I've generally assumed my custom methods should return the same types which are intended to be returned by the default methods, because I get that changing the return value to be different than what other programmers expect is not a nice thing to do! It's best to make code as easy to understand as possible, and avoid things that will confuse the next developer who reads the code.

Interface segregation principle: Earlier in my career I saved old code that was not used in case I wanted it again. Now I delete it and rely on git to resurrect it if it's needed again. The reason is that unused code is literally useless. It wastes developers' time and attention. I haven't written interfaces. But this same concern would motivate me to avoid writing interfaces that require subclasses to implement unused methods, and would motivate me to use other structures like multiple inheritance, mixins/traits, or libraries to implement only the functionality that is needed.

Dependency inversion principle: I don't think I've intentionally tried to make concretions depend only on abstractions.

Do you have experience with TDD? When is it helpful? When is it not?


Test driven development is helpful throughout the whole software development lifecycle. Writing tests first does a good job of connecting an implementation to its user story, design and contract. Test driven development can help keep code and the coder focused on implementing the features that are actually needed (avoiding feature bloat) and promote better code organization, modularization, and separation of concerns, promoting its readability and maintainability. Future users can consult unit tests as a form of documentation-by-example to be sure they understand how each method is intended to be used. TDD provides the developer an immediate feedback loop which helps catch bugs early, immediately after they are created, before deploying to production, and this gives developers, management, and clients more confidence that the software does what it's supposed to do. TDD makes it easier to refactor with confidence you're not breaking something. TDD can save a significant amount of time because it makes it less necessary for a human to manually test the same features over and over. TDD is especially helpful for implementing continuous deployment and delivery, which makes it possible to deploy new features to production faster than some competitors in the same market. Tests can help isolate the problem when applications unexpectedly fail in production after they had been working fine for some time in production beforehand. That kind of failure really does happen(!), and it's challenging to solve those problems without tests, logs, and other forms of instrumentation. After a bug is found, tests can provide assurance the bug is fixed. The more mission-critical a piece of code is to the business, the more it's worth testing to be sure it's still working correctly.

But not everything needs to be tested. Tests aren't needed for exploratory code, temporary prototypes, one-off scripts, or 3rd party libraries, and complete test coverage is less necessary for parts of the code that are not central to the business logic. In practice, I care more to write unit tests for controller methods than for the UI or views. Code that requires lots of stubs and mocks to test can make testing more trouble than it's worth. In the short term, the time needed to write tests must be balanced with the speed at which new features need to be completed. Over the long term, tests should be viewed as a way to reduce the time needed for maintenance (bug-fixing) on the software. The short-term and long-term needs of the business need to be weighed to know how much time should be given to writing tests. Some legacy code cannot be tested easily; newer web application frameworks tend to be set up to be more easily testable.

On the whole, test driven development is more helpful than not.

What new technology have you explored recently and what did you like / dislike about it?


I've used versions 0.5, 1, 2, 3 and the most recent pre-release version to make some hobby apps. Web components' encapsulation and composability are awesome; they are what I always wanted out of ExtJS, jQuery, Backbone and other frontend libraries. Their encouragement to just "use the platform" when you can is excellent; it is the future that's becoming the present, because the platform implements the W3C standards. For example, JavaScript now provides tagged template literals, which are an effective native replacement for JSX. Polymer's team has done a great job providing a stable upgrade path between major versions. Polymer never became popular, and its library-specific 3rd party tooling support isn't strong because it's bleeding-edge technology, but those down sides haven't bothered me much because Polymer is mostly W3C standards, which are undeniably popular and have excellent tooling support, frontend frameworks are beginning to use web components under the hood, and the Polymer team provides very well-designed starter kit apps to show how to put together a deployable app using the best current frontend tools.


I've used AngularJS (version 1) for the past 2 years at work, and am comfortable with it, but find applications made with it unnecessarily complex. One plus is that Angular's tooling support is quite strong.

In order to get comfortable with the newest versions of Angular, I have made some tutorial applications with Angular 7, and appreciate its use of more recent JavaScript and TypeScript features like modules, which make dependency injection a bit easier, and its use of components for organizing an app's functionality, but the application architecture it requires still seems more complex and proprietary than necessary. React, Vue and Polymer are easier to understand and use, which seems to me to be a very important advantage, because it can speed up onboarding new developers, can facilitate faster iterations and can better prevent the accumulation of legacy code which is hard to upgrade to newer versions of its dependencies, so ends up breaking.


That's exactly the problem my current employer needs to solve in the next couple years. I maintain and develop several CodeIgniter 2 applications which require PHP 5.6. In order to run PHP 5.6 on my Ubuntu 18.10 machine, I had previously used the ondrej/php PPA from deb.sury.org which permits installing several versions of PHP in parallel, but because PHP 5.6 is no longer supported (as of Jan. 1, 2019), in order to continue using it I had to hold back (not upgrade) some Ubuntu system packages, and eventually that broke a few other packages. The production servers' system packages will need to be upgraded to remain secure, and to upgrade the system packages, the apps need to be migrated to use PHP 7.x.

So to begin that transition, and prevent my local system from having broken packages, I've begun putting the apps in Docker containers. I like that Docker isolates PHP from my system packages! It will also let us upgrade each legacy PHP 5.6 application to a newer version of PHP individually as we have time, and until then, it will let us run the PHP 5.6 apps on upgraded and so secure servers, and make deployment even more stable than the automated deployment we already have. So I'm getting a Docker setup working to help my coworkers continue to maintain these apps after my contract ends.

This problem my current employer faces has shown me that companies should invest some time in upgrading their stack to newer software in order to prevent their software from becoming unsupported, insecure, broken, or incompatible with newer software, syntax, architectures, paradigms, etc. and the new efficiencies and so business value they can provide. Upgrading too often will slow down new feature development, but not upgrading often enough runs a real risk of the software ceasing to function as a whole.

Last Updated on Thursday, 14 March 2019 17:26
How to get out of crunch mode PDF Print E-mail
News - Web Development
Written by Tim Black   
Thursday, 04 August 2016 22:11

I did some serious reading and thinking about how I might be able to help a startup stay out of crunch mode, and came up with the following recommendations.

Getting out of crunch mode is a challenging transformation, because it requires changing a company's culture, business plan, and software development process, and these things are hard to change.

It is a matter of changing the company's culture. This has to start with the founders. You have to value a work-life balance.[1] One pointed way I saw this put was: you have to value your employees' bodies more than your profit. Benefits: 40 hour weeks improve employees' happiness and productivity[2], and enable the company to hire from a broader pool, including hiring not ONLY junior employees who are smart, productive, ambitious, and perhaps naive about the "churn and burn" process at some startups, but ALSO more senior employees who are domain experts (think Ph.D.s), stable, reliable professionals, and seek a better work-life balance out of wisdom and not merely necessity. You plan to expand your development team, and a broader candidate pool would make it easier to find candidates willing to relocate to [your non-Silicon Valley location].

It is a matter of changing your business plan. The release deadline pressure comes from selling what you don't have. You can transition to selling the features you have, and selling development services. This can be done gradually, and to a greater or lesser extent. Benefits: This reduces deadline pressure. Selling existing features to new customers also increases and diversifies your customer base, and you can leverage this portion of profit to fund new features. You can still contract to create new features for clients, but rather than guaranteeing specific features as the primary deliverables, guarantee your development services for a period of time as the primary deliverable, delivering new features frequently in an agile manner. This is a paradigm shift which could seem impossible to implement because some clients require feature deadlines, but for some clients faster iteration and constant feedback loops actually give them more needed features faster than they would get under feature deadlines.

It is a matter of changing the software development process and feature delivery architecture you use to implement your business plan. You can use branch-by-abstraction, put unfinished features (or alternative/optional/sales-tiered finished features) behind feature flags, implement continuous integration and continuous delivery[3], and so move to more frequent releases and a rolling release cycle. Continuous delivery and frequent releases are harder with software as complex as operating systems[4], but still possible. Benefits: Faster development velocity, shorter time-to-market for new features, so quicker response to competition, and less pressure on release day, because every day is a release day in that stable release candidates are built every day[3]. Some companies say they could not compete in today's market without using continuous delivery.


1. This is the shortest and most blunt article I read which could motivate a founder to change his priorities: http://chadfowler.com/2014/01/22/the-crunch-mode-antipattern.html.

2. Surprisingly, Henry Ford found reducing from 6 to 5 10-hour days actually made his workers produce more per week, and a further reduction to 5 8-hour days brought a further increase in production. This was with assembly line workers; arguably knowledge workers' productivity increases similarly when they are not tired. This and much other related research is mentioned in the paper responding to crunch mode at Electronic Arts at
http://cs.stanford.edu/people/eroberts/cs181/projects/2004-05/crunchmode/index.html. This paper is the best collection of material I found which makes the point that it's best to avoid crunch mode. http://www.igda.org/?page=crunchsixlessons is a good presentation, too.

3. How one company transitioned to continuous delivery: https://www.infoq.com/articles/cd-benefits-challenges. "The engineers commented that they don't feel the same level of stress on the release day that they did previously. That day becomes just another normal day."

4. Startup advisor Jocelyn Goldfein wrote that operating systems require a more regular and probably longer release cycle at http://firstround.com/review/the-right-way-to-ship-software/.

Last Updated on Friday, 28 June 2019 17:06
Two names: CouchDB & Couch App Server PDF Print E-mail
News - Web Development
Written by Tim Black   
Monday, 18 May 2015 04:21

I'm reposting here an This e-mail address is being protected from spambots. You need JavaScript enabled to view it I wrote since it was well-received on the CouchDB marketing list, but its formatting did not display well there.

One of CouchDB's developers asked,

How can we make it that CouchApps strengthen CouchDB and not weaken it by adding confusion?

How do CouchApps fit into the CouchDB story?

I've been thinking about this discussion a little, and wanted to offer a few ideas from the perspective of a user.  In essence, I think the CouchDB community should focus on marketing two concepts named "CouchDB," and "Couch App Server."  This is a middle ground between the two extremes of "R.I.P. CouchApps" and "CouchApps are CouchDB's greatest feature!"  The middle ground is achieved by distinguishing CouchDB and CouchApps, and marketing them as two distinct ideas.  You don't have to stop marketing the idea of CouchApps; just market them as something distinct from CouchDB.  Both can be positively marketed, and succeed or fail on their own merits.  But by distinguishing the two, if CouchApps confuse people, they need not turn people away from CouchDB.  My thoughts and reasons for this are below.

1.  Evaluate the current marketing.  CouchApps are mentioned front and center in the first two paragraphs about CouchDB at http://couchdb.apache.org/.  They are mentioned in concept, though not by name.  They are implied in the slogan, "A Database for the Web" which is explained by those two paragraphs.  If CouchDB's marketing should take a different direction in the future, that marketing ought to deal with the question of whether the promises made in those two paragraphs are true.  Specifically, "CouchDB comes with a suite of features, such as on-the-fly document transformation and real-time change notifications, that makes web app development a breeze."  Does CouchDB "make web app development a breeze," or not?  These two paragraphs are the first ones to change.

2.  Make CouchApps secure.  If multiuser data within CouchApps cannot be properly secured presently because CouchDB does not require clients to send the Host header (I tried to test whether this is the case per this email), then make a config option to make CouchDB require the Host header.  It sounds easy to do, and the Host header is required in HTTP 1.1.  Or create a "default _rewrite path" configuration parameter as Giovanni described.  I expect this would make SmileUpps' CouchApp architecture secure for anyone who wants to use that architecture.

3.  Don't promise CouchApps are easy.  SmileUpps' CouchApp architecture is the only CouchApp architecture I'm aware of which has (almost) implemented document-level ACLs without some proxy server between the browser and CouchDB.  Others may exist; I just don't know about or remember them.  I have 15 years of part-time experience in full-stack web development, and have written two small CouchApps.  It doesn't appear to me that SmileUpps' CouchApp architecture is particularly easy for novices to learn and implement, at least at present.  Perhaps with a Yeoman generator or the like it could become easy.  But its current complexity does not seem to me to "make web app development a breeze," that is, if you want to prevent some users from reading all the data in one database, which is a normal or at least common requirement in web development.  The end result is not good--CouchDB's promise of easy CouchApps will lead novices to build insecure apps.  I wish CouchApps did make web app development a breeze, and would like to see CouchDB still be able to use that promise in its marketing.  But for now, it seems to me that CouchApps shouldn't be marketed to novice developers as an easy point of entry into web development.  CouchApps are rather a way for developers who already know (one of the many ways) how to structure a single-page app to serve that app out of CouchDB, and access CouchDB as the app's database.  It seems to me a developer should learn Backbone or Angular before CouchApps (like the Chatty tutorial assumes:  https://www.smileupps.com/couchapp-tutorial-chatty-read-api).  So, because they generally require 1) knowledge of a client-side framework, 2)  knowledge of CouchApps' file structure and functionality, and 3)  implementing a very specific CouchApp configuration to be properly secured, CouchApps aren't really an entry point into web development.  Instead, CouchApps are a way for non-novice developers to use CouchDB as both a database and an app server.

4.  CouchApps could rise again!  CouchApps' prospect of master-master replication of not only data, but apps, remains attractive to me.  Give users their data, and give them their code!  It's a powerful thing.  What if we all owned Facebook running in PouchDB in our browsers, without a persistent central server?  Diaspora, CouchAppSpora/Monocles, and a bunch of other software have aimed in this direction.  So I don't think it's wise to pull back completely from marketing CouchDB's CouchApp features.  Perhaps they could still be the future.

5.  Reprioritize marketing to serve today's web.  "CouchDB is [still] a database that completely embraces the web."  But the web has changed since CouchDB first embraced it.  Web apps have moved toward offline-first and syncing data to the server via REST.  Web apps don't live on the server anymore.  They live in your phone.  So, as an app developer, I don't need routing or SEF URLs (vhosts/rewrites) on the server; I need routing in the client.  While I still want to be able to query a CouchDB view via HTTP, or maybe even a show or list on occasion to provide an RSS feed, more often I want to query my data locally in PouchDB or something like it.  So it doesn't make sense to me to prominently market CouchDB's "on-the-fly document transformation" anymore.  I still want the feature, but it's not in the core of what I need.  What I need is a database that syncs.  CouchDB's other application server features are nice when I want them.

6.  Market an accurate Venn diagram.  The relationship between CouchDB's CouchApp features and CouchDB's non-CouchApp features is not one where you can divide the features neatly into two mutually-exclusive sets.  Rather, CouchApps use nearly all of CouchDB's features (except maybe replication), and non-CouchApps use a subset of CouchDB's features.  CouchApps may or may not use replication depending on whether they use something like PouchDB, and whether they use replication for deployment.  To put it in terms of my main proposal, future "CouchDB" would be a subset of CouchDB's current features, and "Couch App Server" would be a superset of that future "CouchDB" set.  This makes me think that it is hard for people to grasp what it means to disable the features which support CouchApps, because they too easily believe this means disabling the superset; disabling the whole of CouchDB 1.x's features.  So I think the best way to explain this to new users is to say that CouchDB is a database, and it comes with some extra features which are useful for an application server.  The first two paragraphs at http://couchdb.apache.org/ say as much, but they do not distinguish these two concepts in a way that is obvious and memorable to a newcomer, or that would convince the newcomer that they might want to use CouchDB as a database alone apart from its additional application server features.  The newcomer is left with the impression that the features unique to "Couch App Server" are actually part of the database, but while they are part of CouchDB, yet they are not technically database features; instead, they are web file server and document transformation features.

The way I perceive the natural groupings of CouchDB's features is as follows:

i)  Database:  The database that syncs (views (=indexes), HTTP interface, replication, filters, MVCC, changes feed, authentication)
ii)  [App Server?]:  Web file server (vhosts, rewrites, attachments) & design docs/stored procedures for further document transformation (views, shows, lists, update handlers)

i) is the core set of features, and i) + ii) = CouchApps.  I think this distinction should be displayed textually and graphically on CouchDB's front page.

Interestingly, ii) contains features of the past (server-side apps) and the (maybe) future (client-side apps, deployed through replication).  It would be worth noting this distinction on CouchDB's front page to help newcomers decide to what extent they need Couch's app server features.  It remains possible that someday the present swing toward client-side apps will reverse, and even these server-side features could become more of the future.  So I don't think the marketing should characterize CouchApps as a dying remnant from the past which will not be relevant in the future.

Earlier in CouchDB's lifetime we thought CouchApps using all of ii) were the future.  Now because there is less need for server-side routing and document transformation, the main parts of ii) which might be the future are views (for occasional direct queries over HTTP - yes, I included views in both i & ii), attachments (for serving/replicating your app's files from the database), and lists (for RSS feeds or data syndication/federation through filtered replication).  But it would be good to say on CouchDB's front page that a developer might not need any of the features in ii) today.

So Couch's app server features should not be marketed as extra features for advanced users.  They don't add functionality which every developer will want after they master the basics.  Rather, they are extra features which permit you to write limited server-side logic for your application, if you find you need it.  This is actually a useful point for a newcomer, because while single page applications running in the browser can perform most of the logic we need today, yet (aside from issues of data security and proprietary code, which are the realm of server-side node changes listeners) due to reasons of architecture and efficiency, we are not able to run all our logic in the browser; it's still wise to keep some logic in the database on the server side.  Often we're still dependent on server-side logic, and Couch's app server features can meet that need to a limited extent.

One result of the features in ii) is that your app's server-side and client-side logic can be synced from one server (CouchDB instance) to another.  For example, from your local development machine to the deployment server, or from one deployed application instance to several other nodes all running the same application, but with different filtered sets of data.  This warrants the slogan, "Apps that sync!"  However, that slogan might make people think the syncing in view is between CouchDB and the web browser, which is not what I mean by the slogan.  The apps' syncing is actually done by the features under i), but they are apps because of the features under ii).  So the "Apps that sync" do so because they are in "The database that syncs."

7.  My proposal.  So I propose splitting the features described in the first two paragraphs at http://couchdb.apache.org/, features which are mixed together there under the one name of "CouchDB" without any clear distinction, into two sets of features under two separate names and slogans:

1.  "CouchDB" - the database that syncs!
2.  "Couch App Server" - apps that sync!

Since I (as an app developer) only need CouchDB, market CouchDB most prominently.  But since Couch App Server is nice when I want it, and might be a good method for deploying app updates, market it as a nice but limited set of server-side features your app can use (even to serve your app), which can be secured, and which can be used to deploy app updates, but which you probably don't need if you are using a modern client-side application architecture.  If people want server-side features, this will be a selling point.  If people don't want server-side features, they will appreciate being told that they don't need to research CouchApps.

Because I want Couch App Server's features sometimes (RSS feeds), I think they should be enabled by default.  I don't see why it would be necessary to provide a configuration option to turn them off, but I wouldn't mind if they were turned off.

Though a graphic designer could figure out a better way than this, I envision presenting this distinction between "CouchDB" and "Couch App Server" in a graphic which shows CouchDB as the fundamental layer of Couch's architecture, and Couch App Server as an optional layer on top of CouchDB which includes additional features.  Below the graphic, I envision two columns of text, with CouchDB's description on the left, and Couch App Server's description on the right, and somehow CouchDB being portrayed as the most important of the two.  If you want to hide Couch App Server on the front page, then present only CouchDB on the front page, and provide a link to a page describing "Additional features" which presents the two-layer graphic and two-column descriptions I mentioned above.

If you want to remove the word "Couch" from "Couch App Server," just call it the "App Server."  That was easy.  : )

Last Updated on Tuesday, 30 August 2016 12:10
How I learned web development PDF Print E-mail
News - Web Development
Written by Tim Black   
Tuesday, 11 September 2012 11:04

I wrote the following for a friend in 2008, so some of this info is out of date now.


I've been thinking about how best to introduce you to the world of web development.  I'll describe three main things--the path I've taken to learn web development, the general structure of software, and practical recommendations for where to start.

1.  My path

The general path I've followed in learning web development is below.  All items but the ones marked with an X remain essential for my work today, so could be useful for you to learn.  Of course there are many other unmentioned ways to go about this, and it's good to use the latest books & online materials to your advantage.  You can Google for the keywords below for more info.

Study WebMonkey's Information Architecture Plan
Create summary of web design
Study how to write a business plan, business finances, freelancing
Create web pages using WYSIWYG page editors & editing HTML/CSS/JavaScript by hand
Buy & use the following (now dated) books:
* O'Reilly's Web Design in a Nutshell (overview reference)
* Wrox's Professional PHP Programming (tutorial)
* Sams' Pure JavaScript (exhaustive reference)
Install Xampp (Apache web server, MySQL database, PHP server-side language)
Create pages using PHP & MySQL
Create & maintain sites for various clients - continued through 2004

Learn Python
Learn (centralized) version control - Concurrent Versioning System (CVS), then Subversion (SVN)
Use TikiWiki for my site (a content management system - CMS) - X

Use Joomla (a CMS) for my site & others
Write Joomla components in PHP - X
Write custom AJAX/DHTML

Use (lightweight) "rapid web application frameworks" TurboGears (in Python) & CakePHP (in PHP) - both are patterned after Ruby On Rails (in Ruby), and use the latest programming methods, which are the "best practices."
Use AJAX/DHTML JavaScript toolkits - Scriptaculous, MooTools, jQuery, MochiKit, ExtJS
Learn distributed version control - Bazaar
Pick up lightweight project management tools - Trac/SVN

The path I've taken follows the advances made in web technology over the last 10-15 years (static pages to dynamic database-backed sites to CMSes & rapid web application frameworks), the general hierarchy you might follow to learn web technologies (from basic to more advanced), and the last things in the list are of special value if you need to know (or use) the latest and best technology.

2.  The general structure of software - MVC

It seems to me the best way to introduce you to how a website works is to conceive of the website as if it is a software program or application, which in a sense it is, and group the necessary technologies according to how they function within the program.  One of the best basic paradigms for understanding and organizing a program today is the "Model, View, Controller" paradigm, or MVC.  The Model is where you store your data -- the database.  The View is the part of the program that presents the data visually to you, the user.  The Controller is where you put your programming logic ("business logic") that decides what data to get from the Model (the database) and present to the user through the View, or what to do with information the user sends into the program through the View (often the Controller will write user-submitted data to the database.)  The technologies centrally involved in a website follow this pattern.  To give you a little background, first I'll list some of the things you could put into a website and distinguish which things belong more to the server and which belong more to your particular site:

Web server hardware -  rent space on a host like WebFaction.com for $0-$10/month
Web server software - serves the site's pages to your site's visitors - on Linux servers it's normally Apache; on Windows it's IIS (Internet Information Services)
Database software - MySQL, Postgres, MS SQL Server
Server-side programming language - Perl, PHP, ASP, ASP.net, Java, Python, Ruby, etc.
Domain name - publicly-accessible address for your site's server - register for $20/year with a registrar like GoDaddy.com

Static HTML pages - stored on server, which serves them to site visitors
CSS - (Cascading Style Sheets) - defines the visual layout of HTML elements (dimensions, color, font, etc.)
PHP pages - server-side programming language dynamically generates HTML pages in response to browser's requests, pages are then served by the server
JavaScript - client side code (runs in the web browser) to enrich the user interface - AJAX & DHTML are popular uses
Flash - another client-side language to provide a rich user interface
Content - text, images, audio recordings; stored as files or in the database
Software - ties all the above parts of the site together using a server-side language (PHP, etc.) as the controller - CMS, blog, image gallery, calendar, email app, etc.

Second, you need to understand how all these things are tied together.  They work together as a "stack" of technologies--the "server" parts are foundational for the "website" parts, and the content of your site travels through the stack either from the server to the user, or from the user to the server.  Here's the most common open-source "stack" out there--often called "LAMP" for "Linux, Apache, MySQL, PHP/Perl/Python."

Model - MySQL - data is stored in a MySQL database
Controller - PHP, Apache - the site's software is written in PHP, which reads data from the database & creates an HTML page (inserting data where needed), gives the HTML to Apache, which sends it to the browser.  Or, the PHP code takes data sent by a browser (from a form or a page's link to a URL) & decides what to do with it (maybe write it to the database or a file, send it in an email, etc.)
View - HTML, CSS, JavaScript - the browser interprets HTML (which is just a plain text file) & renders/displays it as a web page.  JavaScript or Flash are used to enrich the user interface here.

This stack is available on most web hosts, and can be set up on your home computer (via XAMPP) to learn PHP& MySQL or to make your programming go faster.  If you don't want to write in a server-side programming language yet, you can create static HTML pages and test them on your home computer without installing/running web server software (Apache).  Just double-click on the HTML file and you can view your site.

There are other stacks.  Some are older and generally more complex, using languages whose syntax is verbose, hence time-consuming and error-prone:

 - C or Perl used to be used as server-side languages
 - Zope for the server, Python for the server-side language (simple), Plone for the CMS (complex), Zope Page Templates for the view (complex)
 - Tomcat for the server & Java for the server-side language

Some are newer, and generally simpler and better.  PHP is easy to learn, and would serve you well, but it's significantly easier to program in Python & Ruby.  Ruby On Rails is a new stack that has popularized newer & better technologies and makes them easy to use.  Right now I'm using a similar stack called TurboGears when I can.

Model - MySQL (or one of several other database backends it supports)
Controller - Python, CherryPy web server
View - XHTML templates, CSS, AJAX toolkits, widgets

3.  Where to start

So, I'd recommend you do the following:

- Make a few static HTML pages to learn HTML.  As needed, use them as a playground to learn CSS & JavaScript.  Use the attached files to get started.  You'll want to edit them in a text editor that gives you syntax highlighting - Notepad++ works well for Windows.
- If you want to learn web programming, install XAMPP & write some PHP pages, create a simple database using phpMyAdmin & use PHP to read from / write to it.
- If you want to learn AJAX, I have 3 simple example files I can send you.
- If you want to build a personal website, install & configure the Joomla CMS (add free plugins as needed) on a web server
- If you want to understand web 2.0 technology in general, get a free FaceBook profile and install a wiki on your web server (or just learn how to use someone else's wiki)
- If you need to understand web 2.0 programming, learn Ruby On Rails (the Rad Rails plugin in the Aptana IDE makes it easy), Django (in Python), or TurboGears

Let me know if you'd like me to give you more info or set up more examples on any of the above and I'll be glad to do so.  If you want me to create a website for you, send me a list of the content & features you will want to have in the site, and I'll write a contract proposal you can modify or accept as the contract.


I wrote the following for another friend around 2010.  It also is somewhat out of date now that applications are moving away from the three-tier architecture where the model and controller, and even the view generation code, runs on the server, to a two-tier architecture where nearly all the code except the database runs on the client.


Historical Background

First, be aware of some major changes that have happened in the history of web development. About every 5 years it seems to me web programming goes through a paradigm shift in the methods used to create a website and organize the programming code in a website. Each new paradigm incorporates some of the previous paradigm, but remains a major change. In 1990, most web pages were static (not interactive), but people handled form submissions using CGI scripts written in Perl. Around 1995, people switched to PHP & ASP, which did a better job of mixing HTML snippets and PHP or ASP (programming commands) code in the same file, but many programmers didn't organize their code well, so it was called "spaghetti code." Around 2000, people started organizing their code better. Around 2005/2006 JavaScript/AJAX libraries became more popular, and the current paradigm of using a "Rapid Web Application Framework" emerged, spearheaded by "Ruby on Rails." Since 2006 I've learned to use something very similar called TurboGears, but in the Python language rather than Ruby. Today things are shifting toward writing single-page applications (they don't refresh the page, but request new data via AJAX) for mobile smartphone touch screens that heavily depend on JavaScript interaction toolkits like Sencha Touch, jQuery Mobile and jQTouch.

Designer-friendly view/layout templates

Django is another Python rapid web application framework, and it was popularized with the story that it enabled its users to quickly modify the appearance of a busy news website (I think in Lawrence, KS). One key way it (and all rapid web application frameworks) made that possible was by organizing the code into a model (which interacts with the database), controllers (which move data from the page to the database or from the database to the page), and views/templates (which present the data in HTML which is then consumed by the web browser.) This is called the "MVC" (model-view-controller) design pattern. Rather than changing all three kinds of code to change the look of the page, as would be necessary with spaghetti code, often all that is necessary for quick layout changes is for a web designer who may be a non-programmer to modify an HTML view/template file.

Invest in newer, but popular and proven, programming languages and methods

So, though much of this may be new to you, I'd highly recommend avoiding investing in the older languages (Perl, PHP, ASP), or older methods (code not organized according to the MVC (or other similar) pattern(s)). It's true that web application frameworks move in and out of popularity quickly (say every 3-5 years?), yet because they encourage organizing code well, it is easier to migrate your code from an old framework to a new one piece by piece when it's organized well according to the MVC pattern. OSQA, for instance, is written in Django. Other software you may consider may be written in PHP or ASP. It can still be wise to pick software written in the older languages, because programming languages never really die--you'll be able to find Perl, PHP and ASP programmers for the next 20 years--but the newer languages Python and Ruby are overtaking the older ones in productivity and so popularity, and so may prove to be a better investment in the end. PHP and ASP web hosts are ubiquitous and cheap, and Python and Ruby web hosts (I use WebFaction, which used to be called Python-Hosting, and have found its tools and support to excel beyond other hosts I've used) are less common, but tend to be more technically competent because they are on the leading edge of developing technologies, and more of their staff and clients tend to be programmers building new software.

With this background, I'll mention why I chose TurboGears rather than Ruby on Rails, Django, CakePHP (in PHP), etc. Ruby is younger than Python, and has had some security problems in its language interpreter, and in 2006 had a less complete set of programming libraries to draw on to do specialized programming tasks. Python had a very complete set of libraries included by default (they say Python comes "batteries included.") Python and Ruby's syntax is simpler, cleaner, briefer, and more like plain English than PHP or ASP. ASP costs money to use because it's owned by Microsoft. Django is monolithic--you use only the Django model library, controller library, and view/template library. TurboGears breaks all functionality into swappable libraries--you can use whatever library you want, but TurboGears recommends a certain set of popular, sane and easy defaults. Because of a fundamental design problem in one of TurboGears' libraries named Pylons (which governs the controller code) that prevented Pylons from being developed further without breaking code that depends on older versions of Pylons, much of TurboGears' community/mindshare is migrating to using a new framework named Pyramid, which mostly plays the same role Pylons played (governing controller code; Pyramid is developed by the Pylons developers as the intentional replacement for Pylons), but also recommends using most of the same libraries used in TurboGears (SQLAlchemy for the model, and Genshi or Mako for the view templates) because they are among the most popular libraries in the Python world for those purposes, so it should be easier to migrate my code from TurboGears to Pyramid than it would be to move from Django to Pyramid.

Programmers' debates over which language or method is best are sometimes described as similar to wars over religion, with the implication that both are matters of preference wrongly turned into matters of life-or-death importance. I don't care what tool a person uses, but I have to choose tools to use myself. I mention the above to you so you can have some wisdom as you go into evaluating in what languages and methods a programmer may recommend you invest, and hopefully make a judgment about whether the investment is wise over the short and long term.

Some other links:



I wrote the following in 2012, so it's up-to-date for now!


I formed the Caney Python User Group (CaneyPUGgies) to teach other people web programming, and make long-term contacts in this area beyond Caney.  The CaneyPUGgies website describes how to get started learning with CaneyPUGgies, the tools we ARE using, and the programming languages and libraries we WERE using:


Our past meeting minutes show the process we've followed, with a detailed record of the code changes we've made:


We WERE writing our hands-on application "Reformed Churches Locator" in TurboGears, which is a rapid web application framework that integrates a large number of best-of-breed libraries and uses the Python language on the server side, and JavaScript on the client/browser side (as do all but Flash or Java web applications).  Then we realized that one feature we need (bidirectional, filtered master-to-master data replication and versioning) would be difficult to write ourselves, so we moved to using the database called CouchDB since it provides that feature by default.  This has required radically reorganizing our code, but also simplifies the code and allows us to reuse most of our JavaScript code.  This is probably a good move, since web applications are increasingly being written in this new 2-layer way (single-page applications whose business logic runs in the browser in JavaScript and communicates to a document-oriented database on the server), rather than in the older three-layer way TurboGears (and the more popular Ruby on Rails) works (MVC - model-view-controller:  model code communicates with the database, view code runs in the browser, controller code runs on the server and moves data between the model and the view.)  So now we have moved to using the JavaScript language exclusively in both the browser and on the server, in CouchApps (a way of organizing the programming code) and CouchDB (a non-relational and document-oriented rather than relational database that runs on the server), and Node.js (an event-driven JavaScript interpreter that runs on the server, for when we need server-side code).

We need to update the CaneyPUGgies website to reflect the languages and libraries we are using now.

To get the tools you need to learn web development with us, you'll need to follow the instructions at http://caneypuggies.alwaysreformed.com/wiki/WebDevEnv.  We can help you with all that at an upcoming meeting; see the meeting schedule at http://caneypuggies.alwaysreformed.com/wiki/PastMeetingMinutes.

Very practically, I recommend learning the basic languages first (see tutorials and cheat sheets at http://caneypuggies.alwaysreformed.com/wiki/CheatSheets ), in this kind of order:

An overview of SQL and how to use a relational database like MySQL (which specific database you learn doesn't matter as much as does the general pattern of how to structure and query data in a relational database)
An overview of CouchDB to understand its different, non-relational paradigm for structuring and querying a database

Last Updated on Tuesday, 23 February 2016 21:55
<< Start < Prev 1 2 3 4 5 Next > End >>

Page 1 of 5