When thinking of the internet and servers, the first type of server that comes to mind is a web server. Not all servers on the internet are actually web servers, but as far as front of mind and siginificance, the web server is way up there.  The previous post covered hosting, and this post covers the server software, A server is simply a computer the can be asked for service, and it the asker has the correct credentials, the server provides the requested service.

This this page reviews web servers software by looking at requirements… from the simplest sites through to more advanced, and looking at the web server technology which can be used to fulfil requirements at each level.

Steps:

• simple web page
  • the most basic web page
• simple web site
  • the content for the simple site
  • basic server version
  • server side templates – removing duplication
  • client side templates – steping past html and simplifying the server
  • CMS: a content management approach to the simple site
  • the basic web server
• Database vs HTML pages for Content
  • database can improve access control
  • matching privelege with expertise
  • simplying content
• Expanding a CMS based website
  • adding a posts/news sections to a CMS website
  • Products, Catalogues and Web Sales
• Uploading Data to the webserver (input)
  • url
  • query string or “get” data
  • “post” data
  • ‘Stateless’ Server Progamming
  • cookies / browser data
• Next Step: Breaking free of Stateless

Simple Web Page

The most basic web page.

HTML for a very basic page can be found here.  So what would a server have to do display this page?  The simplest solution is a server that simply sends files.  If this page was at /index.html then at request to that address would simply return the file for that page.  The webserver would have a folder for the website.  The actual folder for the website would be the ‘root’, the base of the file system for files of the website. The  ‘index.html’ files, as a file in that ‘root’ folder would be sent to the requesting browser.  Then the browser would it self make another request, for the image “images/firefox-icon.png”. This firefox-icon request is for the file from the “images” folder, so under this simple web server model, if there is an “images” folder in the folder holding the site, and in turn a file with the correct name in that folder, then the request for the image from the browser would suceed.  Page rendered.

So far, the server need only serve files from a folder (and its subfolder), serving each request by sending the contents of a file. This is the simplest possible server.

Simple Web Site

The Simple Site Content

As a simplisitc example, consider two pages.  A home page to explain a business, and a contact us page to give address, phone, and store hours.  Each page will have the same  page heading. (to be added if requested: a link to sample html).  The page heading has the store logo, the store name as text and navigiation for the site in the form of named links to both pages.  Alhough this sites is explained with two pages, in considering examples it is best to consider that this could easily expand to more pages with the same concept.

Basic Server Version

These two pages could be built for the simple server as in ‘simple web page’ above, as two individual html files and an image file for the store logo.  All that is required is each web request loads a file.

Taking this approach allows using the file system as a server. This allows testing the pages by simple opening the file in the local file system in a web browser.  No server required.   It also allows hosting the web site on something like github.

The negative of this approach is that the pages break the don’t repeat yourself rule, as they each contain a copy of the html for the heading area of the page and links that make up the navigation bar.    There have been web site editing programs that automatically generate this html repitition from from a single description of how the common areas of the pages should look, but there are also better solutions, which is the reason why this approach has lost favour.

Server Side Templates.

The problem is that some areas of a page are to have the same appearance on several pages.  The first solution to eliminate simply repeating html in each page, is to use server side templates such as mako (example to follow if requrested).

With templates such as mako, pages can avoid the repeated content by simply adding ‘inherit from’, enabling each page to inherit all the repeated html.  There is only one copy of the html that is to appear in both pages, or ‘all’ pages as the site expands.

A genuine web server program is now needed, that for each request checks the file type and runs the mako template engine before serving pages with the “*.htm” extension. (again, an example can be added to this post later if requested).

Client Side Templates.

As discussed in Kotlin DSL Templates, a javascript file (generated by writing Kotlin if desired) can also deliver the repeated content.  This allows each file simply including a lins such as:

As the template is run on the browser, no server functionality is requires to apply the template. This means a client side template will again run as with the ‘basic server version’: almost anywhere and without server code required.

CMS: A Content Management System

With either type of template system, in place of the text of the page being in an html file, the text of each page could be in a database entry.  The advantage of a database entry over a file in a directory, is the separation of coding the web server and the file system of the webserver from content of the website.  The database could even contain entries for different websites.  To enable this, the server program cold examing the page name, together with the domain address, in order to retrieve the releant content for the web domain and page.  The ‘logo’ could be in a settings database entry for each site, and the menu for each page generated automatically from settings for the pages in the database. A value for each page entry in the database indicating as to how/if that page is to appear in a menu.

We have now easily created a server that can manage different sites, and each site with any number of pages.

What we have descirbed is a simplied version of something like wordpress.org. Add a database table for “posts” or “news” and a page to view these “posts” or “news”, and you have a simple webserver for multiple simplified wordpress.org websites.

Database vs HTML pages for content

database provides better access control

access control over the file system of a web server application is complex.  The permission to upload and delete images will differ from permissions relevant to creating new pages. So permissions need to vary depending on file type. Further, new database entries to be added can be checked for complying with rules as to the content of those entries, while giving access to a file system conditional upon edits to files or new files following specific rules for their content is problematic

matching priveledge with expertise

By keeping the text and information content controlled by one group of user (eg.g business owner, marketing), and possible template layouts and the logic of the webserver controlled by a different group (e.g. developers) caters best for situations where, while some people may be members of both groups, there may be people with a role in one of the groups, but not both.  By separating file level access from database access, both groups are supported appropriately.

simplying content

HTML has the complexity from a history of evolving standards, and the need to handle the requirements of every page from the entire world wide web.  The content entry in the database can use a far “simpler to get correct” format such as markdown or even plain text.

Expansions to a CMS website.

Adding a news/posts page to a website.

The “CMS” section covered adding “pages” to a website, with the page name of each page being used to look up the content in the database for that page, and then a template system used to insert that content into the chosen template for the page, together with relevant images and site navigation.

However, a web site will also need some pages that do not conform to the naming of these “pages”.  One catagory could be news, which could have prefix that is not valid for general pages.  The server can then detect this prefix, and with the specific logic for news/posts, use the remainder of the url to hunt through the database entries for posts or news for the appropriate sites, and load that data from the database.  The data can then be loaded into a template to present an html pages of the selected news, or simply all news.

Products Catalogues and Web Sales

The same CMS using a database approach is easily extendable to displaying not just news and/blog posts, but also a product catalogue. An additional table with products, pricing, display pictures and information such as brand, colour etc can be used for products.

Further, such a server again either using multiple databases selected according to domain, or simply a ‘domain’ field to select within a single database, can easily support multiple web sites.

Uploading Data to the Webserver.

So far this page has discussed the webserver recieving a ‘url’ which consists of a domain name and a path name, but there are three further sources of data for the web server appliation:

• the URL
• get data
• post data
• cookies / browser data

The URL.

Already discussed in previous sections, this is again included here for completeness. The URL is all the information in the address bar of a browser, up until the ? (question mark) character, or until the end, where there is no ‘?’ character.  This ‘Uniform Resource Locator’ consists of the host name (for the web, including the domain name) with an optional port number (an optional number following a ‘:’ character, eg. “google.com:80”) , then a ‘/’ character, then a “url path”.  Note the path is not necessarily a path or filename as suggested in the wikipedia page.  The original idea was that this would be a path, and it can be a path, but things have evolved and a server can use ‘path’ information however it chooses.  See this post as an example.

Query String or “Get” data

Get data can be manually entered into the address bar, but is also automatically generated from web form input using the “get” method.  I feel the wikipedia page on query data is a useful explanation.

Query or Get data arrives at the server as the information from the address bar that follows the ‘?’ character, and usually consists of  ‘key value pairs’.  Since this the data is part of the address, this data can easily be saved as bookmark or sent as a link.

Post Data

Query strings have always had limitations, which stem from being appened to the url. At the time of the http 0.9 specification, the only way for forms in web pages to send input data was via the query string, and it became clear this was inflexible.  For http 1.0, post requests were added providing for structured data and far more flexibility in data sent in web requests.   At one time there was even discussion of removing query strings from the specification, but ability to have data present in a bookmark or link became the key argument for keying query strings.

General practice today is to reserve query strings for situations for information used in configuring what is going to be displayed in response to the web request, but no changing anything on the server, and post data sending information to the webserver which will change the information stored on the server.  The means a favourite inquiry can be stored as a link including query data.

Collecting Data On a Webserver Application.

Consider a survey form, completed as three web pages. At minimum, this would requre 4 web requests:

1. request to display: survey page1.html
2. request to store page 1 data and display page2.html
3. request to store page 2 data and display page3.thml
4. request to store page 3 data and display ‘thankyou.html’

IF the form is simple, then the form submiting the survey data could use either the method, get or post.  Both would technically work.

Step 1 will be a simple request for page1.html

Consider step 2. There are three ways of competing this step.

1. as a request to page1.html with data, which instructions to the browse to then load page2.html
2. as a request for url for page2.html with form data which then returns page2.html to the browser
3. as a request for url page1.html with form data which then returns  page2.html to the browser

Option 1 is logically the simplest, but involves two “round trips”, each “trip” send a message from web browser to server, then the server sending back a message to the browser.  The first trip to the server is “here is the form data, save it” and the reply “ok, now load this page” and second trip is “please send me the second page”.

For option 1, on the server, all is simple.  Each page can have its own url.  A request to the page without data to save sends back the page, and a second request to the same url with save data sends back an instruction to load the next page.

For option 2, that extra round trip is eliminated. The ‘submit’ for form in page1, can specify the submit address as the url for page2.  Somewhat confusingly, this means it is logic for processing the url for page2 that saves data for page1.  So page2 logic on the server first saves data for page1, then sends page1.  This means for the web browser the url in the address bar will match the page contents displayed, but the approach makes it harder to handle a situation where page 1 should be redisplayed because questions were not answered.  Sending back a redirect to page1 is a solution, and at least makes the extra round trip only happen when there is a problem.

Now for option 3.  This again saves the extra round trip, but it means the contents of form displayed do match the url, while page1 data is saved when it is sent with a request for page1, the form for page2 is sent back to the browser.  This means the same url returned both page1 and page2, the server deciding on the basis of the data sent, which form to send as a response.  So why not simply use the same url for all the pages, and decide what which page or form to send as a response based on input data?  In fact this can work well, saves every case of double round tripsend every .  Page2 will be displayed with the same url that was used to display page1, because the server decided with page to send based on the form data supplied to the page.

Stateless Server Progamming

For normal coding, a program requests input, recieves a response and the processes that response, knowing what question the response is answering.

A servers recieves each web request as indepenant request, with no knowledge of what has happened previously. Server programs are like Dory in Finding Nemo, they have no idea what has happened in the past. A console program would remember what question is being answered, and process the answer accordingly.  A server has to be told what question is being answered.  With each option above, the data provided with the request reminds the server what has happened previously.  Sometimes it is obvious, sometimes the data in request has to be examined in more detail, but always the goal is to determine the current state from the data provided with the request, because this is the only way for the server to determine the state.

This means every request is potentially the end of a ’round trip’ and contains the answer to a question that the code serving the request did not actually ask.  Consider the 3 options for filling in a form above. The option with the clearest logic to code is option 1, because every request is only receiving an anwer, or sending out a question, and when receiving an anwer, the url clearly notes what answer is being received.  Option 3, the most efficient solution, requires those coding the solution getting their head around processing responses to one question, before then sending out a new and different question, all in the one server request.  Efficient, but potentially confusing.

With option 1, as each page has a different URL, then the URL itself conveys some information about state.  With option 3, state must be determined entirely from the answers to the questions present in the request.  One way to facilitate this with greater clarity is to provide input fields in the form that are no visible on the screen, or do not allow changes, such as “form name”, that can be checked to give clear code determining state from the “anwers” recieived in the request.

Cookies/ Browser Local Storage: State without input forms

In the above examples, the server examines input get/post input data  to determine state. In the simplest possible case,  the request being a post request is sufficient to signal that the state for the URL is currently ‘receiving response’. But in all cases, the code for the URL must determine state, and the more flexible the system, the more possible states. Get “query” and/or “post” data, are normally only present for requests generated from pages with input forms . There are other where it is necessary to determine state without an input form, so another form of input data is required to determine state.

Enter cookies. A clear example of where a server needs to resconstruct state, is for a user who has logged on.  The server needs to be reminded of who the member is, and that their state is “already logged in” and verified, in order to avoid needing every page to have the login data. Cookies are like a reminder note passed back with responses.  Each cookie is a key and value pair (much like query data) that also have a lifetime.  From being received by the browser, each cookie will, until its lifetime ‘expires’ be sent with every request to the same hostname.  So although the server has no inherent memory of the login details being verified the “who are you again?” can be avoided by checking the cookies stored in the browser and sent with each request.

If checking the data stored in a browser such as chrome, it becomes clear that as well as there is also ‘local server data’.  This data the equvilent of cookies, but is not automatically sent to the server, and is instead  accessed by an in browser javascript or webassembly code. This means that client side code can also be designed to access state or stored values, and still be new code for each page, as the code within webpage system ensures.

Next Step: Breaking free of Stateless

The rather confusing to program world of stateless is a result of the original web specification where although the model is ‘client server’, the only method of control from page to page actually oriniates from the server.  Any control from the client, is limited to while staying within the one page.  This works perfectly for one page applications such as google search, but if the application has a more involved user interface as would be in a mobile device delivered through an app,  then bringing the app paradym to the web could be the best solution.

This approach is to consider delivering an app together with a web page, where that app will then use an ajax style interface to the web, sending and receiving web service requests without loading a new page and having the logic reset, everytime a message is sent from or to be received from the web.  The app delivered will remain in control until the logical process launced is complete.

This approach allows for a far more rich user interface, provides more standalone functionality and can deliver a richer experience for apps that move beyond one simple function from the user interface.

This alternative realising the full potential of  ‘web 2.0’.  And puts the server in the true role of serving the app which is placed in control.