gateone.py - Gate One's Main Script

Gate One

Gate One is a web-based terminal emulator written in Python using the Tornado web framework. This module runs the primary daemon process and acts as a central controller for all running terminals and terminal programs. It supports numerous configuration options and can also be called with the --kill switch to kill all running terminal programs (if using dtach--otherwise they die on their own when gateone.py is stopped).

Dependencies

Gate One requires Python 2.6+ but runs best with Python 2.7+. It also depends on the following 3rd party Python modules:

• Tornado 2.2+ - A non-blocking web server framework that powers FriendFeed.

The following modules are optional and can provide Gate One with additional functionality:

• pyOpenSSL 0.10+ - An OpenSSL module/wrapper for Python. Only used to generate self-signed SSL keys and certificates. If pyOpenSSL isn't available Gate One will fall back to using the 'openssl' command to generate self-signed certificates.
• kerberos 1.0+ - A high-level Kerberos interface for Python. Only necessary if you plan to use the Kerberos authentication module.
• python-pam 0.4.2+ - A Python module for interacting with PAM (the Pluggable Authentication Module present on nearly every Unix). Only necessary if you plan to use PAM authentication.

With the exception of python-pam, both the required and optional modules can usually be installed via one of these commands:

user@modern-host:~ $ sudo pip install tornado pyopenssl kerberos

...or:

user@legacy-host:~ $ sudo easy_install tornado pyopenssl kerberos

Note

The use of pip is recommended. See http://www.pip-installer.org/en/latest/installing.html if you don't have it.

The python-pam module is available in most Linux distribution repositories. Simply executing one of the following should take care of it:

user@debian-or-ubuntu-host:~ $ sudo apt-get install python-pam

user@redhat-host:~ $ sudo yum install python-pam

user@gentoo-host:~ $ sudo emerge python-pam

user@suse-host:~ $ sudo yast -i python-pam

Settings

All of Gate One's configurable options can be controlled either via command line switches or by settings in the server.conf file (they match up 1-to-1). If no server.conf exists one will be created using defaults (i.e. when Gate One is run for the first time). Settings in the server.conf file use the following format:

<setting> = <value>

Here's an example:

address = "127.0.0.1;::1;10.1.1.4" # Strings are surrounded by quotes
port = 443 # Numbers don't need quotes

There are a few important differences between the configuration file and command line switches in regards to boolean values (True/False). A switch such as --debug evaluates to "debug = True" and this is exactly how it would be configured in server.conf:

debug = True # Booleans don't need quotes either

Note

The following values in server.conf are case sensitive: True, False and None (and should not be placed in quotes).

Running gateone.py with the --help switch will print the usage information as well as descriptions of what each configurable option does:

root@host:~ $ ./gateone.py --help
Usage: ./gateone.py [OPTIONS]

Options:
  --help                           show this help information
  --log_file_max_size              max size of log files before rollover
  --log_file_num_backups           number of log files to keep
  --log_file_prefix=PATH           Path prefix for log files. Note that if you are running multiple tornado processes, log_file_prefix must be different for each of them (e.g. include the port number)
  --log_to_stderr                  Send log output to stderr (colorized if possible). By default use stderr if --log_file_prefix is not set and no other logging is configured.
  --logging=debug|info|warning|error|none Set the Python log level. If 'none', tornado won't touch the logging configuration.
  --address                        Run on the given address.  Default is all addresses (IPv6 included).  Multiple address can be specified using a semicolon as a separator (e.g. '127.0.0.1;::1;10.1.1.100').
  --auth                           Authentication method to use.  Valid options are: none, api, google, kerberos, pam
  --certificate                    Path to the SSL certificate.  Will be auto-generated if none is provided.
  --command                        Run the given command when a user connects (e.g. '/bin/login').
  --config                         Path to the config file.  Default: /opt/gateone/server.conf
  --cookie_secret                  Use the given 45-character string for cookie encryption.
  --debug                          Enable debugging features such as auto-restarting when files are modified.
  --disable_ssl                    If enabled, Gate One will run without SSL (generally not a good idea).
  --dtach                          Wrap terminals with dtach. Allows sessions to be resumed even if Gate One is stopped and started (which is a sweet feature).
  --embedded                       Doesn't do anything (yet).
  --enable_unix_socket             Enable Unix socket support use_unix_sockets (if --enable_unix_socket=True).
  --https_redirect                 If enabled, a separate listener will be started on port 80 that redirects users to the configured port using HTTPS.
  --js_init                        A JavaScript object (string) that will be used when running GateOne.init() inside index.html.  Example: --js_init="{scheme: 'white'}" would result in GateOne.init({scheme: 'white'})
  --keyfile                        Path to the SSL keyfile.  Will be auto-generated if none is provided.
  --kill                           Kill any running Gate One terminal processes including dtach'd processes.
  --locale                         The locale (e.g. pt_PT) Gate One should use for translations.  If not provided, will default to $LANG (which is 'en_US' in your current shell), or en_US if not set.
  --new_api_key                    Generate a new API key that an external application can use to embed Gate One.
  --origins                        A semicolon-separated list of origins you wish to allow access to your Gate One server over the WebSocket.  This value must contain the hostnames and FQDNs (e.g. https://foo;https://foo.bar;) users will use to connect to your Gate One server as well as the hostnames/FQDNs of any sites that will be embedding Gate One. Here's the default on your system: 'https://localhost;https://yourhostname'. Alternatively, '*' may be  specified to allow access from anywhere.
  --pam_realm                      Basic auth REALM to display when authenticating clients.  Default: hostname.  Only relevant if PAM authentication is enabled.
  --pam_service                    PAM service to use.  Defaults to 'login'. Only relevant if PAM authentication is enabled.
  --pid_file                       Path of the pid file.   Default: /var/run/gateone.pid
  --port                           Run on the given port.
  --session_dir                    Path to the location where session information will be stored.
  --session_logging                If enabled, logs of user sessions will be saved in <user_dir>/<user>/logs.  Default: Enabled
  --session_timeout                Amount of time that a session should be kept alive after the client has logged out.  Accepts <num>X where X could be one of s, m, h, or d for seconds, minutes, hours, and days.  Default is '5d' (5 days).
  --sso_realm                      Kerberos REALM (aka DOMAIN) to use when authenticating clients. Only relevant if Kerberos authentication is enabled.
  --sso_service                    Kerberos service (aka application) to use. Defaults to HTTP. Only relevant if Kerberos authentication is enabled.
  --syslog_facility                Syslog facility to use when logging to syslog (if syslog_session_logging is enabled).  Must be one of: auth, cron, daemon, kern, local0, local1, local2, local3, local4, local5, local6, local7, lpr, mail, news, syslog, user, uucp.  Default: daemon
  --syslog_host                    Remote host to send syslog messages to if syslog_logging is enabled.  Default: None (log to the local syslog daemon directly).  NOTE:  This setting is required on platforms that don't include Python's syslog module.
  --syslog_session_logging         If enabled, logs of user sessions will be written to syslog.
  --unix_socket_path               Run on the given socket file.  Default: /var/run/gateone.sock
  --url_prefix                     An optional prefix to place before all Gate One URLs. e.g. '/gateone/'.  Use this if Gate One will be running behind a reverse proxy where you want it to be located at some sub-URL path.
  --user_dir                       Path to the location where user files will be stored.

Note

Some of these options (e.g. log_file_prefix) are inherent to the Tornado framework. You won't find them anywhere in gateone.py.

File Paths

Gate One stores its files, temporary session information, and persistent user data in the following locations (Note: Many of these are configurable):

File/Directory	Description
authpam.py	Contains the PAM authentication Mixin used by auth.py.
auth.py	Authentication classes.
certificate.pem	The default certificate Gate One will use in SSL communications.
docs/	Gate One's documentation.
gateone.py	Gate One's primary executable/script. Also, the file containing this documentation
i18n/	Gate One's internationalization (i18n) support and locale/translation files.
keyfile.pem	The default private key used with SSL communications.
logviewer.py	A utility to view Gate One session logs.
plugins/	Plugins go here in the form of ./plugins/<plugin name>/<plugin files|directories>
remote_syslog.py	A module that supports sending syslog messages over UDP to a remote syslog host.
server.conf	Gate One's configuration file.
sso.py	A Kerberos Single Sign-on module for Tornado (used by auth.py)
static/	Non-dynamic files that get served to clients (e.g. gateone.js, gateone.css, etc).
templates/	Tornado template files such as index.html.
terminal.py	A Pure Python terminal emulator module.
termio.py	Terminal input/output control module.
tests/	Various scripts and tools to test Gate One's functionality.
utils.py	Various supporting functions.
users/	Persistent user data in the form of ./users/<username>/<user-specific files>
users/<user>/logs	This is where session logs get stored if session_logging is set.
/tmp/gateone	Temporary session data in the form of /tmp/gateone/<session ID>/<files>

Running

Executing Gate One is as simple as:

root@host:~ $ ./gateone.py

Note

By default Gate One will run on port 443 which requires root on most systems. Use --port=(something higher than 1024) for non-root users.

Plugins

Gate One includes support for any combination of the following types of plugins:

• Python
• JavaScript
• CSS

Python plugins can integrate with Gate One in three ways:

• Adding or overriding tornado.web.RequestHandlers (with a given regex).
• Adding or overriding methods (aka "commands") in TerminalWebSocket.
• Adding special plugin-specific escape sequence handlers (see the plugin development documentation for details on what/how these are/work).

JavaScript plugins will be added to the <body> tag of Gate One's base index.html template by way of a single file ({{gateone_js}} below) that is the concatenation of all plugins' JS templates:

<script type="text/javascript" src="{{gateone_js}}"></script>

CSS plugins are similar to JavaScript but instead of being appended to the <body> they are added to the <head> by way of a WebSocket download and some fancy JavaScript inside of gateone.js:

CSSPluginAction: function(url) {
    // Loads the CSS for a given plugin by adding a <link> tag to the <head>
    var queries = url.split('?')[1].split('&'), // So we can parse out the plugin name and the template
        plugin = queries[0].split('=')[1],
        file = queries[1].split('=')[1].split('.')[0];
    // The /cssrender method needs the prefix and the container
    url = url + '&container=' + GateOne.prefs.goDiv.substring(1);
    url = url + '&prefix=' + GateOne.prefs.prefix;
    url = GateOne.prefs.url + url.substring(1);
    GateOne.Utils.loadCSS(url, plugin+'_'+file);
}

There are also hooks throughout Gate One's code for plugins to add or override Gate One's functionality. Documentation on how to write plugins can be found in the Plugin Development docs. From the perspective of gateone.py, it performs the following tasks in relation to plugins:

• Imports Python plugins and connects their hooks.
• Creates symbolic links inside ./static/ that point to each plugin's respective /static/ directory and serves them to clients.
• Serves the index.html that includes plugins' respective .js and .css files.

Class Docstrings

gateone._(string)

Wraps user_locale.translate so we can .encode('UTF-8') when writing to stdout. This function will get overridden by the regular translate() function in __main__

gateone.require_auth(method)

An equivalent to tornado.web.authenticated for WebSockets (TerminalWebSocket, specifically).

gateone.kill_session(session, kill_dtach=False)

Terminates all the processes associated with session and then removes session from the global SESSIONS dict. If kill_dtach is True, will also call kill_dtached_proc() on the terminal processes.

gateone.kill_all_sessions(kill_dtach=False)

Calls kill_session() on all sessions in the SESSIONS dict.

gateone.timeout_sessions(kill_dtach=False)

Loops over the SESSIONS dict killing any sessions that haven't been used for the length of time specified in TIMEOUT (global). The value of TIMEOUT can be set in server.conf or specified on the command line via the session_timeout value. Arguments:

• kill_dtach - If True, will call kill_dtached_proc() on each terminal to ensure it dies.

Note

This function is meant to be called via Tornado's ioloop.PeriodicCallback().

gateone.cleanup_session_logs(users_dir, max_age)

Cleans up all user's session logs older than max_age (timedelta) given the users_dir (string). The session log directory is assumed to be:

users_dir/<user>/logs

class gateone.HTTPSRedirectHandler(application, request, **kwargs)

A handler to redirect clients from HTTP to HTTPS.

get()

Just redirects the client from HTTP to HTTPS

class gateone.StaticHandler(application, request, **kwargs)

An override of tornado.web.StaticFileHandler to ensure that the Access-Control-Allow-Origin header gets set correctly.

set_extra_headers(path)

Adds the Access-Control-Allow-Origin header to allow cross-origin access to static content for applications embedding Gate One. Specifically, this is necessary in order to support loading fonts from different origins.

class gateone.BaseHandler(application, request, **kwargs)

A base handler that all Gate One RequestHandlers will inherit methods from.

get_current_user()

Tornado standard method--implemented our way.

class gateone.DownloadHandler(application, request, **kwargs)

A tornado.web.RequestHandler to serve up files that wind up in a given user's session_dir in the 'downloads' directory. Generally speaking these files are generated by the terminal emulator (e.g. cat somefile.pdf) but it could be used by plugins as a way to serve up temporary files as well.

class gateone.MainHandler(application, request, **kwargs)

Renders index.html which loads Gate One.

Will include the minified version of gateone.js if available as gateone.min.js.

Will encode GATEONE_DIR/static/bell.ogg as a data:URI and put it as the <source> of the <audio> tag inside the index.html template. Gate One administrators can replace bell.ogg with whatever they like but the file size should be less than 32k when encoded to Base64.

class gateone.PluginCSSTemplateHandler(application, request, **kwargs)

Renders plugin CSS template files, passing them the same prefix and container variables used by the StyleHandler. This is so we don't need a CSS template rendering function in every plugin that needs to use {{prefix}} or {{container}}.

gateone.js will automatically load all *.css files in plugin template directories using this method.

class gateone.JSPluginsHandler(application, request, **kwargs)

Combines all JavaScript plugins into a single file to keep things simple and speedy.

_combine_plugins()

Combines all plugin .js files into one (combined_plugins.js)

class gateone.TerminalWebSocket(application, request)

The main WebSocket interface for Gate One, this class is setup to call 'commands' which are methods registered in self.commands. Methods that are registered this way will be exposed and directly callable over the WebSocket.

allow_draft76()

By overriding this function we're allowing the older version of the WebSockets protocol. As long as communications happens over SSL there shouldn't be any security concerns with this. This is mostly to support iOS Safari.

get_current_user()

Mostly identical to the function of the same name in MainHandler. The difference being that when API authentication is enabled the WebSocket will expect and perform its own auth of the client.

open()

Called when a new WebSocket is opened. Will deny access to any origin that is not defined in self.settings['origin'].

on_message(message)

Called when we receive a message from the client.

on_close()

Called when the client terminates the connection.

Note

Normally self.refresh_screen() catches the disconnect first and this method won't end up being called.

pong(timestamp)

Responds to a client 'ping' request... Just returns the given timestamp back to the client so it can measure round-trip time.

authenticate(settings)

Authenticates the client by first trying to use the 'gateone_user' cookie or if Gate One is configured to use API authentication it will use settings['auth']. Additionally, it will accept settings['container'] and settings['prefix'] to apply those to the equivalent properties (self.container and self.prefix).

new_multiplex(cmd, term_id, logging=True)

Returns a new instance of termio.Multiplex with the proper global and client-specific settings.

• cmd - The command to execute inside of Multiplex.
• term_id - The terminal to associate with this Multiplex or a descriptive identifier (it's only used for logging purposes).
• logging - If False, logging will be disabled for this instance of Multiplex (even if it would otherwise be enabled).

term_ended(term)

Sends the 'term_ended' message to the client letting it know that the given term is no more.

new_terminal(*args, **kwargs)

Starts up a new terminal associated with the user's session using settings as the parameters. If a terminal already exists with the same number as settings[term], self.set_terminal() will be called instead of starting a new terminal (so clients can resume their session without having to worry about figuring out if a new terminal already exists or not).

kill_terminal(*args, **kwargs)

Kills term and any associated processes.

set_terminal(*args, **kwargs)

Sets self.current_term = *term* so we can determine where to send keystrokes.

reset_client_terminal(term)

Tells the client to reset the terminal (clear the screen and remove scrollback).

reset_terminal(*args, **kwargs)

Performs the equivalent of the 'reset' command which resets the terminal emulator (among other things) to return the terminal to a sane state in the event that something went wrong (bad escape sequence).

set_title(*args, **kwargs)

Sends a message to the client telling it to set the window title of term to whatever comes out of:

SESSIONS[self.session][term]['multiplex'].term.get_title() # Whew! Say that three times fast!

Example message:

{'set_title': {'term': 1, 'title': "user@host"}}

If force resolves to True the title will be sent to the cleint even if it matches the previously-set title.

Note

Why the complexity on something as simple as setting the title? Many prompts set the title. This means we'd be sending a 'title' message to the client with nearly every screen update which is a pointless waste of bandwidth if the title hasn't changed.

manual_title(*args, **kwargs)

Sets the title of settings['term'] to settings['title']. Differs from set_title() in that this is an action that gets called by the client when the user sets a terminal title manually.

bell(*args, **kwargs)

Sends a message to the client indicating that a bell was encountered in the given terminal (term). Example message:

{'bell': {'term': 1}}

mode_handler(*args, **kwargs)

Handles mode settings that require an action on the client by pasing it a message like:

{
    'set_mode': {
        'mode': setting,
        'bool': True,
        'term': term
    }
}

dsr(term, response)

Handles Device Status Report (DSR) calls from the underlying program that get caught by the terminal emulator. response is what the terminal emulator returns from the CALLBACK_DSR callback.

Note

This also handles the CSI DSR sequence.

_send_refresh(term, full=False)

Sends a screen update to the client.

refresh_screen(*args, **kwargs)

Writes the state of the given terminal's screen and scrollback buffer to the client using _send_refresh(). Also ensures that screen updates don't get sent too fast to the client by instituting a rate limiter that also forces a refresh every 150ms. This keeps things smooth on the client side and also reduces the bandwidth used by the application (CPU too).

If full, send the whole screen (not just the difference).

full_refresh(*args, **kwargs)

Calls self.refresh_screen(*term*, full=True)

resize(*args, **kwargs)

Resize the terminal window to the rows/cols specified in resize_obj

Example resize_obj:

{'rows': 24, 'cols': 80}

char_handler(*args, **kwargs)

Writes chars (string) to term. If term is not provided the characters will be sent to the currently-selected terminal.

write_chars(*args, **kwargs)

Writes message['chars'] to message['term']. If message['term'] is not present, self.current_term will be used.

esc_opt_handler(*args, **kwargs)

Executes whatever function is registered matching the tuple returned by utils.process_opt_esc_sequence().

get_bell()

Sends the bell sound data to the client in in the form of a data::URI.

get_webworker()

Sends the text of our go_process.js to the client in order to get around the limitations of loading remote Web Worker URLs (for embedding Gate One into other apps).

get_style(settings)

Sends the CSS stylesheets matching the properties specified in settings to the client. settings must contain the following:

• container - The element Gate One resides in (e.g. 'gateone')
• prefix - The string being used to prefix all elements (e.g. 'go_')

settings may also contain any combination of the following:

• theme - The name of the CSS theme to be retrieved.
• colors - The name of the text color CSS scheme to be retrieved.
• plugins - If true, will send all plugin .css files to the client.
• print - If true, will send the print stylesheet.

enumerate_themes()

Returns a JSON-encoded object containing the installed themes and text color schemes.

send_message(message)

Sends the given message to the client using the 'notice' WebSocket action at the client.

debug_terminal(*args, **kwargs)

Prints the terminal's screen and renditions to stdout so they can be examined more closely.

Note

Can only be called from a JavaScript console like so...

GateOne.ws.send(JSON.stringify({'debug_terminal': *term*}));

class gateone.ErrorHandler(application, request, status_code)

Generates an error response with status_code for all requests.