Dashboard

Documentation for Dashboard

Shortcuts

Index

Introduction

Web applications often require coding a user account system, organizations, subscriptions and other 'boilerplate' again and again.

Dashboard packages everything web apps need into reusable, modular software. It runs separately to your application so you have two web servers instead of one, and Dashboard fuses their content together to provide a single website or interface for your users. To get started your web app just needs to serve something on / for your guest home page and /home for signed in users.

Dashboard is stateless and designed to scale 'horizontally', that is you can run multiple instances of your Dashboard server in parallel to handle user requests using Heroku, Render, Fly, Digital Ocean's App Platform, etc. Managed hosting is available at Layered Apps.

Your application server can return special HTML attributes and tags to interoperate with the Dashboard server. Your content can be accessible to guests by specifying <html data-auth="false"> and you can serve full-page content by specifying <html data-template="false"> in your HTML. Otherwise the routes will require authentication, and Dashboard will inject the template's header and navigation into the top of the page and merge the head tag's script and CSS references.

You can populate the template's navigation bar by including <template id="navbar"></template> with the links and any other HTML for your menu.

Hosting Dashboard yourself

Dashboard requires NodeJS 16+ be installed.

$ mkdir my-dashboard-server
$ cd my-dashboard-server
$ npm init
$ npm install @layeredapps/dashboard
$ echo "require('@layeredapps/dashboard').start(__dirname)" > main.js
$ node main.js

Customizing Dashboard appearance

Dashboard and module content is provided in HTML pages. Your application server can serve two special CSS files at /public/template-additional.css and /public/content-additional.css to theme the template and pages to match your application design.

/public/template-additional.css
/public/content-additional.css

You can make structural changes to the template, redirect and error pages by serving your own pages from your application server. Dashboard will try to fetch alternative HTML pages from your application server and use its own files if you do not have any:

/template.html
/error.html
/redirect.html

The account and administrator drop-down menus are created from stub HTML files placed in Dashboard, modules, and your application server. To add your own links serve a /menu-account.html and /menu-administrator.html on your application server with the HTML to include.

The account menu is assembled in this order:

  1. Your application server's /menu-account.html
  2. Any activated module's /menu-account.html files
  3. Dashboard's /menu-account.html

The administrator menu is assembled in this order:

  1. Your application server's /menu-administrator.html
  2. Any activated module's /menu-administrator.html files
  3. Dashboard's /menu-administrator.html

Dashboard will try to fetch alternative icons from your application server:

/favicon.ico
/public/favicon.ico
/public/favicon-16x16.png
/public/favicon-32x32.png
/public/apple-touch-icon.png

Customize registration information

By default users may register with just a username and password, both of which are encrypted so they cannot be used for anything but signing in. You can specify some personal information fields to require in an environment variable:

REQUIRE_PROFILE=true
PROFILE_FIELDS=any,combination

These fields are supported by the registration form:

Field Description
full-name First and last name
contact-email Contact email
display-name Name to display to users
display-email Email to display to users
dob Date of birth
location Location description
phone Phone number
company-name Company name
website Website
occupation Occupation

Configuring Dashboard

Dashboard is configured with a combination of environment variables and hard-coded settings in your package.json:

{
    "dashboard": {
        "title": "Title to place in Template",
        "modules: [
            "@layeredapps/organizations",
            "@layeredapps/stripe-connect"
        ],
        "server": [
            "/src/to/script/to/run/receiving/requests.js"
        ],
        "content": [
            "/src/to/script/to/modify/content.js"
        ],
        "proxy": [
            "/src/to/script/to/modify/proxy/requests.js
        ],
        "themeColor": "#abcdef",
        "tileColor": "#abcdef"
    }
}

Server handlers can execute before and/or after a visitor is identified as a guest or user:

module.exports = {
    before: async (req, res) => {
        // req.account is not set
        // req.session is not set
    },
    after: async (req, res) => {
        // req.account may be set
        // req.session may be set
    }
}

Content handlers can adjust the template and page documents before they are served to the user:

module.exports = {
    page: async (req, res, pageDoc) => {
        // adjust page before mixing with template
    },
    template: async (req, res, templateDoc) => {
        // page is now in `src-doc` of application iframe
    }
}

Proxy handlers can add to the headers sent to your application servers:

module.exports = async (req, proxyRequestOptions) => {
    proxyRequestOptions.headers.include = 'something'
}

Provided server, content and proxy handlers

Dashboard comes with some convenience scripts you can add to your package.json:

Type Script path Description
proxy @layeredapps/dashboard/src/proxy/x-account.js Dashboard will bundle the user's Account object in x-account header.
proxy @layeredapps/dashboard/src/proxy/x-csrf-token.js Dashboard will bundle a CSRF token specific to the user session and URL (truncated to ?) that you can place in a hidden form input and compare when the form is submitted to prevent external websites from submitting your forms via "cross-site request forgery". You can alternatively roll your own CSRF protection.
proxy @layeredapps/dashboard/src/proxy/x-profile.js Dashboard will bundle the user's Profile object in x-account header.
proxy @layeredapps/dashboard/src/proxy/x-session.js Dashboard will bundle the user's Session object in x-account header.
server @layeredapps/dashboard/src/server/allow-api-access.js Allows API access from the same domain eg via browser / AJAX.
server @layeredapps/dashboard/src/server/allow-api-access-cross-domain.js Allows API access from third-party domains eg via browser / AJAX.
server @layeredapps/dashboard/src/server/allow-api-requests-from-application.js Allows your application server to query /api/* while Dashboard's API is not publicly shared.
server @layeredapps/dashboard/src/server/allow-api-requests-to-application.js When an unknown /api/* request occurs it will be passed to your application server while Dashboard's API is not publicly shared.
server @layeredapps/dashboard/src/server/check-before-delete-account.js Require users complete steps, such as deleting subscriptions, before deleting their account. Set a CHECK_BEFORE_DELETE_ACCOUNT path such as /check-delete on your Application server, Dashboard will query this API passing ?accountid=xxxxx and you may respond with { "redirect": "/your-delete-requirements" } or { "redirect": false }" to enforce the requirements.
server @layeredapps/dashboard/src/server/fetch-application-server-special-html.js Serve a custom error.html, redirect.html, template.html, menu-account.html or menu-administrator.html from your application server at /FILE_NAME.html. Dashboard will cache this for 60 seconds or an amount you set in CACHE_APPLICATION_SERVER_FILES.
server @layeredapps/dashboard/src/server/fetch-application-server-static-file.js Serve a custom /robots.txt, /favicon.ico or any /public/ files from your application server. Dashboard will cache this for 60 seconds or an amount you set in CACHE_APPLICATION_SERVER_FILES.

Dashboard includes some "private" content scripts that are configured automatically:

Type Script path Description
content @layeredapps/dashboard/src/content/insert-csrf-token.js Adds a CSRF token to Dashboard and module forms, this does not add CSRF tokens to your application's forms.
content @layeredapps/dashboard/src/content/set-form-novalidate.js When NODE_ENV=testing this adds "novalidate" to FORM attributes so input validation (eg required) does not occur so server error messages can be tested.
content @layeredapps/dashboard/src/content/set-form-return-url.js Copies the 'return-url' from URL parameters to the FORM action if there is no pre-existing return-url defined in the FORM action.
server @layeredapps/dashboard/src/server/check-csrf-token.js Compares posted CSRF token to expected CSRF token and renders an error message if they are not matching.
server @layeredapps/dashboard/src/server/check-xss-injection.js Checks input for XSS injections when posting to Dashboard and module forms and APIs. This does not check data posted to your application server since Dashboard does not know if you want HTML in user input.
server @layeredapps/dashboard/src/server/hot-reload.js When HOT_RELOAD=true this forces files to be reloaded on each request so changes can be observed immediately without restarting the server.

Dashboard modules

Dashboard is modular and by itself it provides only the signing in, account management and basic administration. Modules add new pages and API routes for additional functionality.

Name Description
@layeredapps/maxmind-geoip IP address-based geolocation by MaxMind
@layeredapps/organizations User created groups
@layeredapps/maxmind-geoip IP-to-country mapping by MaxMind
@layeredapps/stripe-connect Marketplace functionality by Stripe
@layeredapps/stripe-subscriptions SaaS functionality by Stripe
@layeredapps/oauth Oauth framework for providers
@layeredapps/oauth-github "Sign in with GitHub" using OAuth

Modules are NodeJS packages that you install with NPM:

$ npm install @layeredapps/stripe-subscriptions

You need to notify Dashboard which modules you are using in package.json conffiguration:

"dashboard": {
  "modules": [
    "@layeredapps/stripe-subscriptions"
  ]
}

Access the API

Dashboard and official modules are completely API-driven and you can access the same APIs on behalf of the user making requests. You perform GET, POST, PATCH, and DELETE HTTP requests against the API endpoints to fetch or modify data. You can use a shared secret APPLICATION_SERVER_TOKEN to verify requests between servers, both servers send it in an x-application-server-token header.

By default the API is not accessible, you can allow total access to it with an ENV variable:

"dashboard": {
  "server": [
    "@layeredapps/dashboard/src/server/allow-api-access.js"
  ]
}

Or enable requests from your application server with a server script handler:

"dashboard": {
  "server": [
    "@layeredapps/dashboard/src/server/internal-api-requests.js"
  ]
}

Dashboard can also forward API requests to your own /api/ routes:

"dashboard": {
   "server": [
     "@layeredapps/dashboard/src/server/forward-api-requests.js"
   ]
}

This example fetches the user's session information using NodeJS, you can do this with any language:

const sessions = await proxy(`/api/user/sessions?accountid=${accountid}`, accountid, sessionid)

const proxy = util.promisify((path, accountid, sessionid, callback) => {
    const requestOptions = {
        host: 'dashboard.example.com',
        path: path,
        port: '443',
        method: 'GET',
        headers: {
            'x-application-server': 'application.example.com',
            'x-application-server-token': process.env.APPLICATION_SERVER_TOKEN,
            'x-accountid': accountid,
            'x-sessionid': sessionid
        }
    }
    const proxyRequest = require('https').request(requestOptions, (proxyResponse) => {
        let body = ''
        proxyResponse.on('data', (chunk) => {
            body += chunk
        })
        return proxyResponse.on('end', () => {
            return callback(null, JSON.parse(body))
        })
    })
    proxyRequest.on('error', (error) => {
        return callback(error)
    })
    return proxyRequest.end()
  })
}

Storage backends

Dashboard uses the Sequelize library and is compatible with PostgreSQL, MySQL, MariaDB and SQLite. Support for Microsoft SQL Server and IBM DB2 is possible but has issues (MSSQL, DB2). During development and testing you can use SQLITE in RAM by not setting a DATABASE_FILE.

Storage Environment variables
SQLITE STORAGE=sqlite
DATABASE=dashboard
DATABASE_FILE=/path/file.sqlite
POSTGRESQL STORAGE=postgresql
DATABASE_URL=postgres:/...
MariaDB STORAGE=mariadb
DATABASE_URL=mysql://...
MYSQL STORAGE=mysql
DATABASE_URL=mysql://...
MSSQL STORAGE=mssql
DATABASE_URL=Server=....
DB2 STORAGE=db2
DATABASE_URL=Server=...

Dashboard modules are able to use their own storage settings:

$ SUBSCRIPTIONS_STORAGE=postgresql \
  SUBSCRIPTIONS_DATABASE_URL=postgres://localhost:5432/subscriptions \
  ORGANIZATIONS_STORAGE=mysql \
  ORGANIZATIONS_DATABASE_URL=mysql:// \
  CONNECT_STORAGE=mssql \
  CONNECT_DATABASE_URL=... \
  node main.js

Read replication is supported for PostgreSQL, MariaDB, MySQL, MSSQL and DB2 by setting STORAGE_REPLICATION and then defining 1+ read-only databases.

STORAGE_REPLICATION=true
READ_DATABASE_URL1= 
READ_DATABASE_URL2=

Connection pooling limits can be defined including on a per-module configuration:

MAX_CONNECTIONS=3
IDLE_CONNECTION_LIMIT=10000
ORGANIZATIONS_MAX_CONNECTIONS=10
ORGANIZATIONS_IDLE_CONNECTION_LIMIT=60000

Storage caching

You can complement your storage backend with optional caching, either using RAM if you have a single instance of your Dashboard server, or Redis if you need a cache shared by multiple instances of your Dashboard server.

You can optionally use Redis as a cache, this is good for any storage on slow disks or for scaling Dashboard with multiple instances.

$ STORAGE_CACHE=redis \
  REDIS_URL=redis://.... \ # use dedicated redis for cache
  node main.js

If you have a single Dashboard server you can cache within memory:

$ CACHE=node \
  node main.js

Usage metrics

By default usage metrics will be stored in your Dashboard database. You can specify a Redis database to use instead.

You can optionally use Redis as a cache, this is good for any storage on slow disks.

$ STORAGE_METRICS=redis \
  REDIS_URL=redis:/.... \
  node main.js

Single and multiple Redis servers

You can specify a single Redis server used for both caching and metrics:

$ STORAGE_METRICS=redis \
  STORAGE_CACHE=redis \
  REDIS_URL=redis:/.... \
  node main.js

You can specify separate servers:

$ STORAGE_METRICS=redis \
  STORAGE_METRICS_REDIS_URL=redis:/.... \
  STORAGE_CACHE=redis \
  STORAGE_CACHE_REDIS_URL=redis:/.... \
  node main.js

XSS and CSRF protection

Dashboard protects its own forms and module forms from "cross-site scripting" attacks and "cross-site request forgery", these attacks are essentially if someone enters javascript as a form value that is then executed when it is written on a page, or if they tried to submit a form from their website using an iframe.

Data that is posted to your application server is not restricted in case you have valid reasons to receive values that would not pass Dashboard's checks.

You can use the @layeredapps/dashboard/src/proxy/x-csrf-token.js to receive a CSRF token for your own forms, and then when your forms are submitted you can check the posted value against the header value. These tokens are generated based on the URL, a random value assigned to the session, and the user's session and account identifiers.

For XSS there are many language-specific libraries you can use to sanitize input. Dashboard is written in NodeJS and uses Yahoo's XSS-filters module.

Logging

By default Dashboard does not have any active console.* being emitted. You can enable logging with LOG_LEVEL containing a list of valid console.* methods.

$ LOG_LEVEL=log,warn,info,error node main.js

Override Dashboard's logging by creating your own log.js in the root of your project:

module.exports = (group) => {
  return {
    log: (message) => {
    },
    info: (message) => {
    },
    warn: (message) => {
    }
  }
}

Creating modules for Dashboard

A module is a NodeJS application with the same folder structure as Dashboard. When Dashboard starts it scans its own files, and then any modules specified in the package.json to create a combined sitemap of UI and API routes. You can browse the official modules' source to see examples.

$ mkdir my-module
$ cd my-module
$ npm install @layeredapps/dashboard --no-save
# create main.js to start the server
# create index.js optionally exporting any relevant API
# add your content
$ npm publish

The "--no-save" flag is used to install Dashboard, this prevents your module from installing a redundant version of Dashboard when it is being installed by users.

When your module is published users can install it with NPM:

$ npm install your_module_name

Modules must be activated in a web app's package.json:

dashboard: {
    modules: [ "your_module_name" ]
}

These paths have special significance:

Folder Description
/src/www Web server root
/src/www/public Static assets served quickly
/src/www/account User account management pages
/src/www/account/YOUR_MODULE/ Your additions (if applicable)
/src/www/administrator Administration pages
/src/www/administrator/YOUR_MODULE/ Your additions (if applicable)
/src/www/api/user User account management pages
/src/www/api/user/YOUR_MODULE/ Your additions (if applicable)
/src/www/api/administrator Administration APIs
/src/www/api/administrator/YOUR_MODULE/ Your additions (if applicable)
/src/www/webhooks/YOUR_MODULE/ Endpoints for receiving webhooks (if applicable)

Content pages may export before, get for rendering the page and post methods for submitting HTML forms. API routes may export before, get, post, patch, delete, put methods. If specified, the before method will execute before any verb.

Guest-accessible content and API routes can be flagged in the HTML or NodeJS:

# HTML
<html auth="false">

# NodeJS API route
{ 
    auth: false,
    get: (req) = > {

    }
}

Content can occupy the full screen without the template via a flag in the HTML or NodeJS:

# HTML
<html template="false">

# NodeJS page handler
{ 
    template: false,
    get: (req, res) = > {

    }
}

Testing

Dashboard's test suite covers the API and the UI. The API tests are performed by proxying a running instance of the software. The UI tests are performed with puppeteer remotely-controlling Chrome to browse a running instance of the software and Mocha to execute the tests. A docker-compose.yml is provided to create databases.

$ npm run test-sqlite
$ npm run test-sqlite-node-cache
$ npm run test-sqlite-redis-cache
$ npm run test-mysql
$ npm run test-mysql-node-cache
$ npm run test-mysql-redis-cache
$ npm run test-mariadb
$ npm run test-mariadb-node-cache
$ npm run test-mariadb-redis-cache
$ npm run test-postgresql
$ npm run test-postgresql-node-cache
$ npm run test-postgresql-redis-cache
$ npm run test-mssql
$ npm run test-mssql-node-cache
$ npm run test-mssql-redis-cache
$ npm run test-db2
$ npm run test-db2-node-cache
$ npm run test-db2-redis-cache

Support and contributions

If you have encountered a problem post an issue on the appropriate Github repository.

If you would like to contribute check Github Issues for ways you can help.

Icons

Icons are obtained via FlatIcon and copyright their respective owners:

License

This software is licensed under the MIT license, a copy is enclosed in the LICENSE file. Included icon assets are licensed separately, refer to the icons/licenses folder for their licensing information.

Copyright (c) 2022 Ben Lowry

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Top of page