Web APIs

Overview

Teaching: 15 min
Exercises: 5 min

Questions

• How do Web APIs facilitate communication between software systems over the internet?

• What role does HTTP play in Web API interactions?

• How are requests and responses structured to exchange data effectively?

Objectives

• Understand Web API communication via client-server requests and responses.

• Explore HTTP’s role in standardizing Web API interactions and security with HTTPS.

• Learn the structure of HTTP requests and responses for effective data exchange.

As described previously, Web APIs enable communication between different software systems over the web. Typically, a request is made by a client, which could be an application on your local laptop, to the API server that understands and processes the request and retrieves the data requested and sends it back as a response to the client.

How APIs work

This process is governed by a set of predefined rules and protocols that ensure seamless communication, regardless of the diversity in programming languages and hardware platforms involved. The cornerstone protocol in Web API interactions is HTTP (Hypertext Transfer Protocol), which outlines how messages are formatted and transmitted over the web, and how web servers and browsers should respond to various commands.

To make effective use of web APIs, we need to understand a little more about how the Web works than a typical Web user might. This lesson will focus on clients—computers and software applications that make requests to other computers or applications, and receive information in response. Computers and applications that respond to such requests are referred to as servers.

World Wide Web

At its core, the initial World Wide Web concept brought together three key ideas:

1. The use of HTML (Hypertext Markup Language) documents which could contain hyperlinks to other documents (or different parts of the same document). These could reference documents located on any web server in the world.
2. That every file on the world wide web would have a unique URL (Uniform Resource Locator).
3. The Hypertext Transfer Protocol (HTTP) that is used to transfer data from the web server to the requesting client.

URLs

A URL (also sometimes known as a URI or Uniform Resource Indicator) is the complete web address used to access a specific resource on the internet. It indicates the protocol (e.g., https), domain name, and sometimes the path to a specific resource. It consists of two or three parts: the protocol followed by ://, the server name or IP address and optionally the path to the resource we wish to access. For example the URL http://carpentries.org means we want to access the default location on the server carpentries.org using the HTTP protocol. The URL https://carpentries.org/contact/ means we want to access the contact location on the carpentries.org server using the secure HTTPS protocol.

URL query String

A query string is part of a URL that contains data to be sent to the server for filtering or requesting specific information. It starts after the ? character and consists of key-value pairs separated by &. For example:

https://www.youtube.com/watch?v=s7wmiS2mSXY&t=1m45s

• Key-value pairs:
  • v=s7wmiS2mSXY tells YouTube which video to display.
  • t=1m45s specifies the timestamp at which the video should start (1 minute and 45 seconds in).

Protocols, HTTP & HTTPS

You may (or may not) have wondered how it is that different web browsers, written independently by different companies and running on different operating systems, are able to talk to the same web servers using the same addresses, and get the same web pages back. This is because all web browsers implement the HyperText Transfer Protocol, or HTTP.

A protocol is nothing more than a system of rules that allow for communication between computers (or other devices). Much like a (human) language, it defines rules and syntax that when all parties follow, allow information to be transmitted from one device to another.

HTTPS is a protocol closely related to HTTP; it follows many of the same conventions as HTTP, particularly in the way client and server code is written, but includes additional encryption to ensure that untrusted third parties can’t read or modify data in transit.

Requests and responses

The two main objects in HTTP are the request and the response. Each HTTP connection is initiated by sending a request, and is replied to with a response. Both the request and response have a header, that defines metadata about what is requested and what is included in the response, and both can also have a body, containing data.

An HTTP request typically includes:

• HTTP Method: This indicates the action to be performed on the resource and includes verbs like GET (retrieve data), POST (submit data), PUT (update data), DELETE (remove data), among others.

• Headers: These provide essential information about the request or the client itself, such as content type, authentication details, etc.

• Body: Not present in all types of requests, the body contains data sent by the client to the server, commonly seen in POST or PUT requests.

An HTTP response from a server typically consists of the following key components:

• Status Line: Includes the HTTP version, a status code, and a status message. The status code is a three-digit number that indicates the outcome of the request, with common codes including:

200 OK: The request was successful, and the response body contains the requested data. 404 Not Found: The requested resource could not be found on the server. 500 Internal Server Error: A generic error message indicating that something went wrong on the server.

• Headers: Provide metadata about the response, similar to request headers. Important response headers to understand are:

• Content-Type: Specifies the type of data in the response body (e.g., application/json, text/html).

• Content-Length: The length of the response body in octets (8-bit bytes).

• Body: Contains the data being sent back to the client. This could be the requested resource, confirmation of an action taken, or an error message. The presence and format of the body depend on the request type and the status code. For example:

In a GET request for a webpage, the body would contain the HTML of the page. In a POST request that submits data (like a form submission), the response body might contain a confirmation message or the details of the created resource.

HyperText?

Both HTTP and HTML refer to HyperText. This was a popular buzzword in the 1990s, and refers to the Web’s ability to include not only text, but also cross-references in the form of links (hypertext links, or hyperlinks) to other documents stored elsewhere, which the user can immediately access.

While this seems entirely obvious and second-nature today, it was revolutionary when it was first introduced, hence the name appearing prominently in technologies that supported it.

Web APIs like OpenAI’s API

• Remote Resources: Web APIs are hosted on remote servers. Applications interact with these APIs over the network, usually via HTTP requests.
• Complex Operations: Web APIs often expose functionality that would be impractical to implement locally, either due to the complexity of the operations or the need for access to large, constantly updated datasets. OpenAI’s API, for instance, provides access to advanced AI models trained on vast amounts of data.
• Latency and Scalability: Calls to web APIs are subject to network latency and depend on the API provider’s infrastructure for scalability and performance. This can introduce delays but also offers capabilities far beyond what could be locally hosted.
• Examples: OpenAI for AI-powered text generation, Google Maps API for map-related features, Twitter API for social media interactions.

The OpenAI API leverages the cloud-hosted GPT models to generate chat completions, a task requiring significant computational resources and access to a pre-trained model.

Key Points

• Web APIs enable client-server interactions over the internet through a structured exchange of requests and responses, allowing for seamless data retrieval and submission.

• The Hypertext Transfer Protocol (HTTP) is crucial for Web API operations, outlining how messages are formatted and transmitted, ensuring consistent communication across different platforms.

• In HTTP, the communication involves requests initiated by clients (with methods like GET, POST) and responses from servers, each containing headers and potentially a body with the relevant data.

• HTTPS adds a layer of encryption to HTTP, enhancing security by protecting data in transit, making it essential for sensitive transactions.