Lisp Web Toolkit

Object-Oriented Lisp Systems for Rapid Web Application Development

GitHub
https://github.com/xh4/web-toolkit Star
Status
Build Status
Author
Xiangyu He <[email protected]>
Update
June 11, 2020

1 Get Started

1.1 Get Lisp Web Toolkit

To get the source code of Lisp Web Toolkit, use the following Git command:

git clone https://github.com/xh4/web-toolkit.git

To make the systems discoverable by ASDF, put the source code into directories described in Configuring ASDF to find your systems.

Lisp Web Toolkit currently runs on SBCL, Clozure CL and LispWorks on Linux and macOS, LispWorks on Windows.

1.2 Load a System

Evaluate the following code to load a system:

(asdf:load-system :wt.uri)
(asdf:load-system :wt.json)
(asdf:load-system :wt.html)
(asdf:load-system :wt.css)
(asdf:load-system :wt.javascript)
(asdf:load-system :wt.http)
(asdf:load-system :wt.websocket)

Or load all systems by evaluating:

(asdf:load-system :wt)

2 URI

The WT.URI system implements URI parser and constructor, with support for UTF-8 characters, IPv6 addresses and query parameters handling. It utilizes recursive descent parser combinators to provide a concise implementation that is close to the definition in RFC 3986 Uniform Resource Identifier (URI): Generic Syntax.

2.1 Class uri

Packageuri
standard-object
Metaclassstandard-class

A class which represents a URI.

A URI has the following syntax and components:

URI syntax diagram
URI syntax diagram

2.2 Function uri

Packageuri
Syntax (uri uri-designator)
(uri &rest uri-components)
(uri base-uri relative-uri)
(uri uri-designator &rest uri-components)
Arguments
URI-DESIGNATOR
A URI instance or a string which represents a URI.

URI-COMPONENTS
A property list whose key can be :scheme, :userinfo, :host, :port, :path, :query or :fragment.

BASE-URI
A URI designator.

RELATIVE-URI
A URI designator.

The "All-in-One" function to parse, merge, update and construct a URI.

Parse a URI:

(uri "https://xh.coobii.com")#<URI "https://xh.coobii.com">

Construct a URI:

(uri :scheme "https" :host "xh.coobii.com")#<URI "https://xh.coobii.com">

Merge two URIs:

(uri "https://xh.coobii.com/foo" "/bar")#<URI "https://xh.coobii.com/bar">

Update a URI:

(uri "https://xh.coobii.com" :query '("foo" "bar"))#<URI "https://xh.coobii.com?foo=bar">

2.3 Function uri-string

Packageuri
Syntax (uri-string uri-designator)
(uri-string &rest uri-components)
(uri-string base-uri relative-uri)
(uri-string uri-designator &rest uri-components)
Arguments
URI-DESIGNATOR
A URI instance or a string which represents a URI.

URI-COMPONENTS
A property list whose key can be :scheme, :userinfo, :host, :port, :path, :query or :fragment.

BASE-URI
A URI designator.

RELATIVE-URI
A URI designator.

Do the same thing as the function uri, render the final URI to a string.

(uri-string :scheme "https" :host "xh.coobii.com")"https://xh.coobii.com"
(uri-string "https://xh.coobii.com/foo" "/bar")"https://xh.coobii.com/bar"
(uri-string "https://xh.coobii.com" :port 443)"https://xh.coobii.com:443"
(uri-string :query '("foo" "bar" "goo" "gle"))"foo=bar&goo=gle"
(uri-string :query '(("foo" . "bar") ("goo" . "gle")))"foo=bar&goo=gle"

2.4 Accessor uri-scheme

(uri-scheme "https://xh.coobii.com")"https"
(prog1
    (setq uri (uri "https://coobii.com"))
  (setf (uri-scheme uri) "wss"))
#<URI "wss://coobii.com">

2.5 Accessor uri-userinfo

(uri-userinfo "https://[email protected]")"xh"
(prog1
    (setq uri (uri "https://[email protected]"))
  (setf (uri-userinfo uri) "someone"))
#<URI "https://[email protected]">

2.6 Accessor uri-host

(uri-host "https://coobii.com")"coobii.com"
(uri-host "http://127.0.0.1")"127.0.0.1"
(uri-host "http://[3ffe:2a00:100:7031::1]")"3ffe:2a00:100:7031::1"
(prog1
    (setq uri (uri "https://coobii.com"))
  (setf (uri-host uri) "xh.coobii.com"))
#<URI "https://xh.coobii.com">

2.7 Accessor uri-port

(uri-port "https://coobii.com:443")443
(uri-port "https://coobii.com")NIL
(prog1
    (setq uri (uri "https://coobii.com"))
  (setf (uri-port uri) 443))
#<URI "https://coobii.com:443">

2.8 Accessor uri-path

(uri-path "https://coobii.com")NIL
(uri-path "https://coobii.com/foo/bar")"/foo/bar"
(prog1
    (setq uri (uri "https://coobii.com/coca"))
  (setf (uri-path uri) "/cola"))
#<URI "https://coobii.com/cola">

2.9 Accessor uri-query

(uri-query "https://coobii.com?foo=bar")(("foo" . "bar"))
(uri-query "https://coobii.com?foo=bar" :type :hash-table)#<EQUAL Hash Table{1}>
(prog1
    (setq uri (uri "https://coobii.com"))
  (setf (uri-query uri) '(("foo" . "bar"))))
#<URI "https://coobii.com?foo=bar">

2.10 Accessor uri-fragment

(uri-fragment "https://coobii.com/#header")"header"
(prog1
    (setq uri (uri "https://coobii.com/#header"))
  (setf (uri-fragment uri) "footer"))
#<URI "https://coobii.com/#footer">

3 JSON

The WT.JSON system implements JSON encoder and decoder based on ECMA-404 The JSON Data Interchange Standard. It distinguishes null, false and [] from Lisp's NIL thus supports identical transformation between JSON values. It provides object constructor and accessor to build and access nesting JSON objects.

3.1 Class true

Packagejson
standard-object
Metaclassstandard-class

A class which represents JSON's true.

3.2 Constant true

The constant value represents JSON's true.

3.3 Function true

Packagejson

The function turn it's argument to the constant true or NIL.

(true t)#<TRUE >
(true 42)#<TRUE >
(true nil)NIL

3.4 Class false

Packagejson
standard-object
Metaclassstandard-class

A class which represents JSON's false.

3.5 Constant false

The constant value represents JSON's false.

3.6 Function false

Packagejson

The function turn it's argument to the constant false or the argument itself.

(false nil)#<FALSE >
(false '(1 2 3))(1 2 3)
(false 42)42
(false (object "foo" "bar"))(object "foo" "bar")

3.7 Class null

Packagejson
standard-object
Metaclassstandard-class

A class which represents JSON's null.

3.8 Constant null

The constant value represents JSON's null.

3.9 Function null

Packagejson

The function turn it's argument to the constant null or the argument itself.

(null nil)#<NULL >
(null '(1 2 3))(1 2 3)
(null 42)42
(null (object "foo" "bar"))(object "foo" "bar")

3.10 Class array

Packagejson
standard-object
Metaclassstandard-class

A class which represents a JSON array.

Array is printed in it's constructing form.

3.11 Function array

Packagejson

The function to construct a JSON array from a sequence.

(array)(array)
(array 1 2 3)(array 1 2 3)

3.12 Class object

Packagejson
standard-object
Metaclassstandard-class

A class which represents a JSON object.

Object is printed in it's constructing form.

3.13 Function object

Packagejson

The function to construct a JSON object.

(object "name" "XH" "email" "[email protected]")(object "name" "XH" "email" "[email protected]")
(object "answers" '(1 2 3))(object "answers" (array 1 2 3))

3.14 Function value

Packagejson

Get the inner value of certain JSON types.

(value true)T
(value false)NIL
(value null)NIL
(value (array 1 2 3))(1 2 3)
(value (array))NIL
(value 42)42
(value "string")"string"

3.15 Accessor get

(get (object "foo" "bar") "foo")"bar"
(get
 (object "name" (object "first-name" "Xiangyu" "last-name" "He"))
 "name"
 "first-name")
"Xiangyu"
(prog1
    (setq object
          (object
           "name"
           (object "first-name" "Xiangyu" "last-name" "He")))
  (setf (get object "name" "first-name") "X"
        (get object "name" "last-name") "H"))
(object "name" (object "first-name" "X" "last-name" "H"))
(prog1
    (setq object (object "answers" (array 0 0 0)))
  (setf (get object "answers" 1) 42))
(object "answers" (array 0 42 0))

3.16 Function encode

Packagejson
(encode t)"true"
(encode true)"true"
(encode nil)"false"
(encode false)"false"
(encode null)"null"
(encode (array))"[]"
(encode '(1 2 3))"[1,2,3]"
(encode #(1 2 3))"[1,2,3]"
(encode (object))"{}"
(encode (object "name" "XH" "email" "[email protected]"))"{\"name\":\"XH\",\"email\":\"[email protected]\"}"
(encode (object "answers" '(1 2 3)))"{\"answers\":[1,2,3]}"

3.17 Function decode

Packagejson
(decode "true")#<TRUE >
(decode "false")#<FALSE >
(decode "null")#<NULL >
(decode "[]")(array)
(decode "[1,2,3]")(array 1 2 3)
(decode "{}")(object)
(decode "{\"name\":\"XH\",\"email\":\"[email protected]\"}")(object "name" "XH" "email" "[email protected]")
(decode "{\"answers\":[1,2,3]}")(object "answers" (array 1 2 3))

4 HTML

The WT.HTML system implements HTML constructor, parser and serializer based on recent version of HTML Living Standard. It utilizes the Document Object Model (DOM) provided by WT.DOM.

4.1 Class document

Packagehtml
dom:document
dom:node
dom:non-element-parent-node
dom:document-or-shadow-root
dom:parent-node
standard-object
Metaclassstandard-class

A class which represents a HTML Document.

4.2 Function document

Packagehtml
Syntax (document &optional child)
Arguments
CHILD
An element.
Values
DOCUMENT
A document node.

The function to construct a document node.

(document)#<DOCUMENT >
(document (html (head) (body)))#<DOCUMENT >

4.3 Class element

Packagehtml
dom:element
dom:node
dom:parent-node
dom:non-document-type-child-node
dom:child-node
standard-object
Metaclassstandard-class

A class which represents a HTML Element.

4.4 Element Constructors

An element constructor is a function to construct a HTML element. It accepts attributes and children as arguments.

(h1 "The title")#<H1  {"The title"}>
(img
 :src
 "https://www.wikipedia.org/portal/wikipedia.org/assets/img/Wikipedia-logo-v2.png")
#<IMG >
(section (h1) (h1 (h2 (h3))) (h1))#<SECTION  {3}>

The html package exposes the following element constructors: a abbr address area article aside audio b base bdi bdo blockquote body br button canvas caption cite code col colgroup data datalist dd del details dfn dialog div dl dt em embed fieldset figcaption figure footer form h1 h2 h3 h4 h5 h6 head header hgroup hr html i iframe img input ins kbd label legend li link main mark menu menuitem meta meter nav noscript object ol optgroup option output p param picture pre progress q rb rp rt rtc ruby s samp script section select slot small source span strong style sub summary sup table tbody td template textarea tfoot th thead title tr track u ul var video wbr

4.5 Class text

Packagehtml
dom:text
dom:character-data
dom:node
dom:non-document-type-child-node
dom:child-node
standard-object
Metaclassstandard-class

A class which represents a HTML Text.

4.6 Function text

Packagehtml
Syntax (text &optional data)
Arguments
DATA
A string as the data of the text node.
Values
TEXT
A text node.

The function to construct a text node.

(text)#<TEXT NIL>
(text "Hello, world")#<TEXT "Hello, world">

When passing string as child to element constructor, the string is automatically converted to a Text. (h1 "title") is the same as (h1 (text "title")).

4.7 Function serialize

Packagehtml
Syntax (serialize root &optional stream)
Arguments
ROOT
A node (document, element or text).

STREAM
A stream or NIL.
Values
OUTPUT
NIL or a string, depends on the STREAM argument.

Serialize a HTML node to a stream.

(serialize (div (span "1") (span "2")))"<div><span>1</span><span>2</span></div>"

5 CSS

The WT.CSS system implements CSS constructor, tokenizer, parser and serializer based on specifications introduced in CSS Snapshot 2018.

5.1 Class declaration

Packagecss
standard-object
Metaclassdeclaration-class

A class which represents a CSS declaration. Declarations are further categorized as "properties" or "descriptors".

5.2 Class property

Packagecss
declaration
standard-object
Metaclassdeclaration-class

A class which represents a CSS property.

5.3 Function property

Packagecss
Syntax (property name value)
Arguments
NAME
A string for the name of the property.

VALUE
A string for the value of the property.
Values
PROPERTY
A property instance.

The function to construct a property.

(property "margin" "5px 10px")#<MARGIN "5px 10px">

5.4 Property Constructors

A property contructor is a function to constructor a property.

(margin "10px auto")#<MARGIN "10px auto">
(border-color "rgb(173, 255, 47)")#<BORDER-COLOR "rgb(173, 255, 47)">

The CSS package exports the following property constructor symbols:

Color

color opacity

Box

margin margin-top margin-right margin-left margin-bottom margin-trim padding padding-top padding-right padding-left padding-bottom

Sizing

width height min-width min-height max-width max-height box-sizing line-height vertical-align

Display

position top right bottom left

Logical

float clear resize caption-side

Overflow

overflow overflow-x overflow-y text-overflow

Text

text-transform white-space word-break line-break hyphens overflow-wrap word-wrap text-align text-align-all text-align-last text-justify word-spacing letter-spacing text-indent hanging-punctuation text-decoration-line text-decoration-style text-decoration-color text-decoration text-decoration-position text-emphasis-style text-emphasis-color text-emphasis text-emphasis-position text-shadow text-decoration-skip-ink

Font

font font-family font-weight font-stretch font-style font-size

Background

background background-color background-image background-repeat background-attachment background-position background-clip background-origin background-size box-shadow shadow

Border

border-color border-top-color border-right-color border-bottom-color border-left-color border-style border-top-style border-right-style border-bottom-style border-left-style border-width border-top-width border-right-width border-bottom-width border-left-width border border-top border-right border-bottom border-left border-radius border-top-left-radius border-top-right-radius border-bottom-right-radius border-bottom-left-radius border-collapse

Flexbox

flex flex-direction flex-wrap flex-flow order flex-grow flex-shrink flex-basis justify-content align-items align-self align-content

User Interface

outline outline-width outline-style outline-color outline-offset user-select cursor caret caret-color caret-shape nav-up nav-right nav-down nav-left appearance

List

list-style-image list-style-type list-style-position list-style marker-side counter-reset counter-increment counter-set

Content

content

Transition

transition transition-property transition-duration transition-timing-function transition-delay

Transform

transform translate scale rotate transform-style perspective perspective-origin translate backface-visibility

Animation

animation animation-name animation-duration animation-timing-function animation-iteration-count animation-direction animation-play-state animation-delay animation-fill-mode

Masking

clip clip-path clip-rule mask mask-image mask-mode mask-repeat mask-position mask-clip mask-origin mask-size mask-composite mask-border-source mask-border-mode mask-border-slice mask-border-width mask-border-outset mask-border-repeat mask-border mask-type

Pointer Events

pointer-events touch-action

Filter Effects

backdrop-filter

5.5 Class rule

Packagecss
standard-object
Metaclassstandard-class

A class which represents a CSS rule. Rules are further categorized as qualified rules or at-rules.

5.6 Class qualified-rule

Packagecss
rule
standard-object
Metaclassstandard-class

A class which represents a CSS qualified rule.

5.7 Function rule-prelude

Packagecss
Syntax (rule-prelude rule)
Arguments
RULE
A qualified rule.
Values
PRELUDE
A list of tokens.

Get the prelude part of a qualified rule.

5.8 Function rule-block

Packagecss
Syntax (rule-block rule)
Arguments
RULE
A qualified rule.
Values
BLOCK
A simple block or a list of tokens.

Get the block part of a qualified rule.

5.9 Class style-rule

Packagecss
qualified-rule
rule
standard-object
Metaclassstandard-class

A subclass of qualified rule.

5.10 Function rule-selector

Packagecss
Syntax (rule-selector rule)
Arguments
RULE
A style rule.
Values
SELECTOR
A string or a list of strings.

Get the selector of a style rule.

5.11 Function rule-declarations

Packagecss
Syntax (rule-declarations rule)
Arguments
RULE
A style rule.
Values
DECLARATIONS
A list of declarations.

Get the declarations of a style rule.

5.12 Function rule

Packagecss
Syntax (rule selector &rest properties)
Arguments
SELECTOR
A string or a list of strings.

PROPERTIES
A list of properties.
Values
RULE
A style rule.

This function constructs a style rule.

(rule '("h1" "h2" "h3") (color "black") (line-height "1.5"))#<STYLE-RULE ("h1" "h2" "h3") {2}>

5.13 Function tokenize

Packagecss
Syntax (tokenize source)
Arguments
SOURCE
A string or a stream.
Values
TOKENS
A list of tokens.

This funtion runs the tokenizer on the SOURCE, return a list of TOKENS.

(tokenize "body { background: #fff }")(#S(IDENT-TOKEN :VALUE "body") #S(WHITESPACE-TOKEN) #S(LEFT-CURLY-BRACKET-TOKEN) #S(WHITESPACE-TOKEN) #S(IDENT-TOKEN :VALUE "background") #S(COLON-TOKEN) #S(WHITESPACE-TOKEN) #S(HASH-TOKEN :TYPE NIL :NAME "fff") #S(WHITESPACE-TOKEN) #S(RIGHT-CURLY-BRACKET-TOKEN))

5.14 Function parse-rules

Packagecss
Syntax (parse-rules source)
Arguments
SOURCE
A string or a stream.
Values
RULES
A list of rules.

This funtion runs the parser on the SOURCE, return a list of qualified rules. The body (block) of the rule is given as a list of tokens, to further parse it as declarations, use parse-declarations.

(parse-rules "body { background: #fff }")(#<QUALIFIED-RULE PRELUDE: (#S(IDENT-TOKEN :VALUE "body") #S(WHITESPACE-TOKEN)) BLOCK: #S(SIMPLE-BLOCK :ASSOCIATED-TOKEN #S(LEFT-CURLY-BRACKET-TOKEN) :VALUE (#S(WHITESPACE-TOKEN) #S(IDENT-TOKEN :VALUE "background") #S(COLON-TOKEN) #S(WHITESPACE-TOKEN) #S(HASH-TOKEN :TYPE NIL :NAME "fff") #S(WHITESPACE-TOKEN)))>)

5.15 Function parse-declarations

Packagecss
Syntax (parse-declaration source)
Arguments
SOURCE
A simple block, a string or a stream.
Values
DECLARATIONS
A list of declarations.

This funtion runs the parser on the SOURCE, return a list of declarations. The value of a declaration is given as a list of tokens, to get the string value, call serialize-tokens on the tokens.

(let ((rule (first (parse-rules "body { background: #fff }"))))
  (parse-declarations (rule-block rule)))
(#<DECLARATION NAME: "background" VALUE: (#S(HASH-TOKEN :TYPE NIL :NAME "fff"))>)
(parse-declarations "color: black; margin: 5px")(#<DECLARATION NAME: "color" VALUE: (#S(IDENT-TOKEN :VALUE "black"))> #<DECLARATION NAME: "margin" VALUE: (#S(NUMBER-TOKEN :VALUE 5) #S(IDENT-TOKEN :VALUE "px"))>)

5.16 Function serialize-tokens

Packagecss
Syntax (serialize-tokens tokens &optional stream)
Arguments
TOKENS
A list of tokens.

STREAM
A stream or NIL.
Values
OUTPUT
NIL or a string.

Serialize a list of TOKENS to a STREAM. This function is useful to unparse a rule's prelude (selector) or a declaration's value.

(let ((rule (first (parse-rules "body h1 { color: #333 }"))))
  (serialize-tokens (rule-prelude rule)))
"body h1 "
(let ((declaration
       (first (parse-declarations
               "box-shadow: 0 0 0 0.2rem rgba(255, 193, 7, 0.5)"))))
  (serialize-tokens (declaration-value declaration)))
"0 0 0 0.2rem rgba(255, 193, 7, 0.5)"

6 JavaScript

WT.JAVASCRIPT implements JavaScript parser and serializer based on ECMAScript 2016 Language Specification (ECMA-262 7th Edition) with support for JSX syntax extension. It utilizes syntax tree components from the ESTree project.

7 HTTP

WT.HTTP provides HTTP functionality based on RFC 7231 Hypertext Transfer Protocol (HTTP/1.1), together with implementation of server and client.

8 WebSocket

WT.WEBSOCKET provides WebSocket functionality based on RFC 6455 The WebSocket Protocol, with interface inspired by JSR 356, Java API for WebSocket. The system is tested against the Autobahn WebSocket Testsuite.