Web Client Programming

John "Scooter" Morris

April 11, 2017

Introduction

• The web has become integral to the analysis and presentation of scientific data
• The web provides critical resources
  • Access to key databases (NCBI, Ensemble, PDB)
  • Access to computational services (BLAST, BLAT, DaliLite)
• Scientific users often expect to be able to access data and analysis from the web
  • The challenge is to integrate it all for our users and colleagues
  • That is the focus of web programming

Web Programming

• Web programming is a very broad topic
• We're going to break it up into five parts:
  1. HTTP basics
  2. Client programming:
     • HTML Forms
     • Javascript
  3. Server as a client:
     • Downloading and analyzing content from the web
  4. Server programming:
     • CGI
  5. Putting it all together:
     • AJAX
• Our focus is on web programming
• See the Software Carpentry lecture for a more general discussion of Internet programming

The Hypertext Transfer Protocol

Figure 1: HTTP Request Cycle

HTTP Request Line

Figure 2: HTTP Request

Headers

• An HTTP header is a key/value pair
  • "Accept: text/html"
  • "Accept-Language: en, fr"
  • "If-Modified-Since: 16-May-2005"
• Unlike a dictionary, a key may appear any number of times
  • So a request can specify that it's willing to accept several types of content

Body

• The body is any extra data associated with the request
  • Used with web forms, to upload files, etc.
• Must be a blank line between the last header and the start of the body
  • Signals the end of the headers
  • Forgetting it is a common mistake
• The "Content-Length" header tells the server how many bytes to read
• Note: there's no magic in any of this
  • An HTTP request is just text—any program that wants to can create them or parse them

HTTP Response

Figure 3: HTTP Response

HTTP Response Codes

Questions on HTTP?

Client Programming

• Elements of web client programming:
  • HTML layout, HTML forms, and CSS to provide the basic interface
  • CGI on the server to handle and process the data
  • Javascript to provide some interactivity and client-side processing
  • AJAX (Asynchronous Javascript and XML) to provide a more sophisticated experience
• You have already learned some HTML and CSS...
• ...now let's figure out how to get information from the user

HTML Forms

• HTML forms allow users to enter text, choose items from lists, etc.
  • Not nearly as sophisticated as desktop interfaces
  • Although programmers are doing more every day (particularly using Javascript)
• Basic model:
  1. Browser presents a form with elements to collect the data
  2. User fills out the data and submits the form
  3. Form data is sent to the server for processing
  4. Server replies with a new HTML page that provides any errors or results

Creating Forms

• Create a form using a <form>…</form> element
  • action attribute specifies the URL to send data to
  • method attribute specifies the type of HTTP request to send
    • Usually "GET" for simple HTML forms
    • Use "POST" for more complicated forms and for loading data
  • Example:

    <form action="myscript.cgi" method="get"></form>

  • Can include any HTML within the <form>…</form> tags:

    <form action="myscript.cgi" method="get">
      <div class="entry">
        <span class="header">First Name: </span>
        <input name="first_name" type="text"/>
      </div>
      <div class="entry">
        <span class="header">Last Name:</span>
        <input name="last_name" type="text"/>
      </div>
    </form>

Form Fields

• To provide for user input, use form fields
• Four form field elements:
  • select to let users choose values from a list
  • button for basic push buttons
  • textarea for large text input
  • input used for all other form fields (checkboxes, file uploads, radio buttons, single-line text input)
• All form fields take a name attribute that can be used on the server to access the form data
• Most form fields take a disabled attribute to disable the element
• input elements do not generally have an end tag (all controls and data are in attributes)

Form Fields: <select>

• <select/> elements to let users choose values from a list
• List items specified using embedded <option/> elements
  • selected attribute indicates which option is selected
  • value attribute will pass a value to the server
  • label attribute specifices a shorter label
• Attributes:
  • Use multiple attribute to allow for multiple selection
  • Use size to set the number of visible options in the list

<form action="myscript.cgi" method="get">
  <select multiple="multiple" name="state" size="3">
    <option value="AL">Alabama</option>
    <option value="AZ">Arizona</option>
    <option value="AR">Arkansas</option>
    <option value="CA" selected="selected">California</option>
    <option value="CO">Colorado</option>
    …
  </select>
  <select name="state">
    <option value="AL">Alabama</option>
    <option value="AZ">Arizona</option>
    <option value="AR">Arkansas</option>
    <option value="CA" selected="selected">California</option>
    <option value="CO">Colorado</option>
    …
  </select>
</form>

Form Fields: <button>

• <button/> elements provide a general push-button mechanism
• Use text between start and end tags to provide rich text and graphics on buttons
• Attributes:
  • Use type attribute to set the button type:
    • button is a "normal" button
    • reset is a "reset" button
    • submit is a "submit" button, which causes the form and all of its data to be sent to the server
  • Use value attribute to set the button value

<form action="myscript.cgi" method="get">
  <button value="clickme" name="button1" type="button"><b>Click</b> <i>me</i>!</button>
</form>

Form Fields: <textarea>

• <textara/> elements provide a general text input mechanism
• Text between start and end tags provides initial values (instructions?)
• Attributes:
  • Use rows and cols to specify the size
  • Use readonly attribute to make the field read only

<form action="myscript.cgi" method="get">
  <textarea rows="6" cols="20" name="textinput">Enter text to be uploaded</textarea>
</form>

Form Fields: <input/>

• Input is the main form field type
• Content between start and end tags will be ignored (use <input/> version)
• Attributes:
  • The type attribute indicates the input type:
    • button: a push button
    • checkbox: an on/off checkbox
    • file: a file browser to upload a file
    • hidden: no visible element (used to pass data to server)
    • image: a button with an image
    • password: a text entry, but with password echo
    • radio: radio buttons. Radio buttons with the same name are grouped
    • password: a text entry, but with password echo
    • submit and reset create buttons to submit the form, or re-set the data to initial values
    • text: a one-line text entry box
  • The size attribute is used to specify the width of the field
  • The maxlength attribute is used to specify the length in characters of an input field
  • The src attribute is used to specify the URL to load for button images
  • The checked attribute is used to specify that an input element is checked

A Simple Form

<html>
  <body>
    <form>
      <p>Sequence: <input type="text"/>
      Search type:
      <select>
        <option>Exact match</option>
        <option>Similarity match</option>
        <option>Sub-match</option>
      </select>
      </p>
      <p>Programs: 
      <input type="radio" name="alg">
        FROG (version 1.1)
      </input>
      <input type="radio" name="alg">
        FROG (2.0 beta)
      </input>
      <input type="radio" name="alg">
        Bayes-Hart
      </input>
      </p>
      <p>
        <input type="button" value="Submit Query"/>
        <input type="button" value="Reset"/>
      </p>
    </form>
  </body>
</html> 

HTML5 Form Extensions

• HTML5 adds new attributes:
  • placeholder attribute:

    <input type=text placeholder="(555) 555-5555">

    used to provide sample text in an input field.
  • maxlength attribute:

    <input type=text maxlength="10">

    used to indicate the maximum length of text input
  • autofocus attribute:

    <input autofocus type=text>

    used to automatically focus the cursor
  • min, max, and step attributes:

    <input type=number min=0 max=100 step=5>

    used to restrict the minimum, maximum and step values for numeric input

HTML5 Form Extensions

• list attribute (and datalist attribute):

  <input type=text list=planetdata>
  <datalist id=planetdata>
  	<option value="Earth">
  	<option value="Venus">
  	<option value="Mars">
  </datalist>

  used to provide a list of options within a text box
• autocomplete attribute:

  <form autocomplete=off>

  used to disable autocompletion on a form.
• Other new attributes:
  • accept: restrict file inputs to certain types based on mime type
  • multiple: allow file inputs to include multiple files
  • required: this field is required (can not be blank)
  • pattern: input must match a regular expression pattern
  • formnovalidate, formaction, formmethod, formtarget, formenctype various extensions to forms

HTML5 Form Extensions

• HTML5 adds new input types:
  • email: accept only e-mail addresses
  • tel: accept phone numbers
  • url: accept url's
  • date, datetime, datetime-local, month, week, time: various date and time input
  • color: input color
  • number: input a number
  • range: input a number within a range (usually using a slider)
  • search: input a search term
• And new elements:
  • meter: outputs a scale bar based on a value in a range
  • progress: a progress bar
  • output: display results of a calculation from form elements
  • keygen: generates a key pair

HTML5 Form Extensions

• Many of these extensions are not yet implemented
• Those that are, may only be implemented in a subset of browsers
• See The Current State of HTML5 Forms for more information

Questions on Forms?

Javascript

• Designed to execute in the browser
• Syntax is similar to 'C' or 'Java'
  • Just remember to end your statements with semi-colons
• JavaScript is a complete scripting language
• JavaScript language details are beyond our scope
• But you'll need it for your project, so...
• JavaScript resources:
  • W3Schools Tutorial: (www.w3schools.com/JS/)
  • Mozilla developer center "re-introduction to JavaScript": (http://developer.mozilla.org/en/docs/A_re-introduction_to_JavaScript)
  • And, of course, the 1,000,000's of web pages with embedded JavaScript examples

Why Javascript?

• JavaScript provides a way to provide dynamic content
  • A JavaScript statement like this: document.write("<h1>"+name+"</h1>"); can write a variable text into an HTML page
• JavaScript can react to events
  • A JavaScript function can be set to execute when something happens, like whan a page has finished loading, or when a user clicks on an HTML element
• JavaScript can be used to validate data
  • A JavaScript can be used to validate form data before it is submitted to a server. This saves the server from extra processing and provides more immediate, and (and arguably, better feedback) to the user
• JavaScript can read and write HTML elements
  • JavaScript can read and change the content of an HTML element

Basic syntax

• JavaScript is a dynamically typed langage, like Python
  • Don't need to worry about variable types:

    var x=1;
    var x1="someText";
    x="someOtherText";

    are all legal JavaScript statements
• Whitespace doesn't matter in JavaScript, unlike Python
  • Blocks are enclosed by braces: {}:

    var i=0;
    for (i = 0; i <= 10; i++)
    {
      document.write("The number is "+i);
      document.write("<br/>");
    }

• Functions are defined with the function statement:

  function displayGene(name, start, protein)
  {
    alert("Gene "+name+" starts at "+start+
          " and translates to the sequence: "+protein);
  }

Getting your script to execute

• Scripts can be embedded in the doucment in a number of ways
• Within the page:
  • JavaScript statements can appear anywhere within the <body> within <script/> tags:

    <script type="text/javascript"> document.write("Hello World!"); </script>

• In the document <head>
  • Similar to what we did with CSS, we can embed our JavaScript directly in the document <head> surrounded by <script/> tags.
• Loaded from an external file:
  • JavaScript functions, data, and statements can be loaded from an external file by providing a src attribute to the script tag:

    <script type="text/javascript" src="js/myFuncs.js" />

Getting your script to execute (events)

• As an action (event) handler:
  • HTML has a number of defined "events" that are expressed as attributes to HTML tags. The target of the event is a script:

    <p>This is a 
    <span style="color:green;border:thin blue solid;" onclick="alert('Hi!');">clickable</span> 
    word</p> 

    Try me!

    This is a clickable word

Getting your script to execute (events)

• As an action (event) handler:
  • Can also call your own function -- this is how form validation can be handled:

    <html>
    <body>
      <form>
        <script type="text/javascript">
          function validate()
          {
            var input = document.getElementById("number");
            var number = Number(input.value);
            if (number < 1 || number > 10 || isNaN(number)) {
              alert("Number must be between 1 and 10");
              input.value = "";
              return false;
            }
            return true;
          }
        </script>
        Enter a number between 1 and 10: 
           <input type="text" id="number" name="number" onchange="validate();"/>
      </form>
    </body>
    </html>

  Try It!

HTML Events

Javascript and the DOM

• The HTML DOM is exposed through the document object
• DOM methods provide access to the HTML elements:
  • document.getElementById(), document.getElementsByName(), document.getElementsByTagNameNS(), document.getElementsByClassName()
  • Note that only getElementById() is singular. All others will return an array of elements
  • Once you've found the element, you can look at it's properties:

    var x = element.innerHTML // the HTML contains within this element
    var x = element.style // the style information for this element
    var x = element.className // the element's class
    var x = element.attributes // the element's attributes

  • Elements also support the getElementBy methods. This allows you to walk the HTML tree, if you need to
• DOM references:
  • mozilla developer center Gecko DOM Reference
  • W3Schools XML DOM Tutorial

Javascript and the DOM

• DOM methods also provide a means to add to the HTML or to modify HTML elements:
  • document.createELementNS(), document.createAttribute(), and element.appendChild() provide the tools need to add to an HTML document
  • Many of the element properties are mutable. So, to change the class of an element, you could just change the className property.
• Playing with classes (and the Gettysburg Address) classTest.html

<html>
<head>
        <title>Class change test</title>
        <link type="text/css" rel="stylesheet" href="classTest.css"/>
        <script type="text/javascript" src="classTest.js" ></script>
</head>
<body>
<h1 class="header" id="header" onclick="changeClass(this);">Playing with classes</h1>

<p class="plain" onclick="changeClass(this);">Four score and seven years ago ...</p>
<p class="plain" onclick="changeClass(this);">Now we are engaged ...</p>
<p class="plain" id="final" onclick="changeClass(this);">
  But, in a larger sense, we can not dedicate ...</p>
<p class="hidden" id="citation">Gettysburg Address<br/>
Abraham Lincoln<br/>
November 19, 1863</p>

</body>
</html>

Javascript and the DOM (Example CSS)

h1.header {
        text-align: left;
        font-family: serif;
        color: black;
}

h1.title {
        text-align: center;
        font-family: sans-serif;
        color: green;
}

p.plain {
        margin-left: 0em;
        margin-right: 0em;
        font-style: normal;
}
p.quote {
        margin-left: 5em;
        margin-right: 5em;
        font-style: italic;
}
p.hidden {
        visibility: hidden;
}
p.visible {
        visibility: visible;
        float: right;
        color: blue;
        font-style: italic;
        margin-right: 5em;
}

Javascript and the DOM (Example JS)

// This function will change the class of the passed element
// Note that we deal with the header specially, and also
// handle the citation depending on the class of the final paragraph
function changeClass(element) {
        // Get the current class name of the element that called us
        var class = element.className;
        // Get the ID of the element that called us
        var id = element.id;

        // Did the header call us?
        if (id == "header") {
                // Yes, switch our class
                if (class == "header") {
                        element.className = "title";
                } else if (class == "title") {
                        element.className = "header";
                }
                return;
        }
        // Make sure it's a paragraph
        if (element.tagName == "P" || element.tagName == "p") {
                // Yes, switch it
                if (class == "plain") {
                        element.className = "quote";
                        // Is it the final paragraph?
                        if (id == "final")
                                // Yes, switch the citation
                                document.getElementById("citation").className = "visible";
                } else if (class == "quote") {
                        element.className = "plain";
                        // Is it the final paragraph?
                        if (id == "final")
                                // Yes, switch the citation
                                document.getElementById("citation").className = "hidden";
                }
        }
}

Other Javascript Objects

• JavaScripts supports the following built-in objects:
  • Types: Array, Boolean, Date, Number, String
  • Math
  • RegExp
  • Top-level functions: decodeURI(), decodeURIComponent(), encodeURI(), encodeURIComponent(), escape(), eval(), isFinite(), isNaN(), Number(), parseFloat(), parseInt(), String(), and unescape()
  • Browser interface objects: Window, Navigator, Screen, History, and Location
  • DOM objects, some of which are: Document, Form, Option, Select, and Style
  • Various HTML elements are exposed as objects, but I think it's better to work through the DOM methods directly

Debugging client code

• Tools:
  • Firefox
    • Most standard-compliant browser
    • Firefox "Tools→Web Developer→Error Console" will really help!
    • Firefox "Tools→Web Developer→Inspector" and "Tools→Web Developer→Debugger"
    • If you don't have Firebug installed -- install it!
• Technique:
  • Separate CSS and JS from HTML file, unless you have < 10-15 lines
  • Start with HTML & CSS - no JS
  • Before coding, search the web for examples
  • Incrementally add and debug JS
  • alert() is a useful, quick debugging aid

Client-side Programming

• Questions?
• Overwhelmed?

Assignment

• Before class tomorrow:
  1. As discussed yesterday, put together the major use cases for your web site
  2. Choose one use case to prototype, and put together a low-fidelity prototype (paper, whiteboard, etc.)
  3. Develop a non-functional HTML prototype of your site:
     • See http://bmi219.rbvi.ucsf.edu/shared/setup.html for information about how plato directories should be set up for revision control and web access
     • Set up your git repository following example in http://bmi219.rbvi.ucsf.edu/shared/example.html
     • Put your HTML in /home/socr/b/bmi219/?/team/member
     • Put your Javascript in /home/socr/b/bmi219/year/?/team/member
     • Put your CSS style sheets in /home/socr/b/bmi219/year/?/team/member
  4. Test your site by going to: http://www.rbvi.ucsf.edu/bmi219/team/member/file.html
  5. Where "team" should be replaced by "a", "b", or "c" as is appropriate for your team and "member" should be replaced by the name of the repository for a member of the team
• During class tomorrow:
  1. Add Javascript to your prototype to validate input, expand fields, and in general present the appropriate options to the user
• Questions?