utils.py - Supporting Functions

Gate One utility functions and classes.

exception utils.UnknownFacility

Raised if string_to_syslog_facility is given a string that doesn't match a known syslog facility.

exception utils.MimeTypeFail

Raised by create_data_uri if the mimetype of a file could not be guessed.

exception utils.SSLGenerationError

Raised by gen_self_signed_ssl if an error is encountered generating a self-signed SSL certificate.

exception utils.ChownError

Raised by recursive_chown if an OSError is encountered while trying to recursively chown a directory.

utils.noop(*args, **kwargs)

Do nothing (i.e. "No Operation")

utils.write_pid(path)

Writes our PID to path.

utils.read_pid(path)

Reads our current PID from path.

utils.remove_pid(path)

Removes the PID file at path.

utils.shell_command(cmd, timeout_duration=5)

Resets the SIGCHLD signal handler (if necessary), executes cmd via commands.getstatusoutput(), then re-enables the SIGCHLD handler (if it was set to something other than SIG_DFL). Returns the result of getstatusoutput which is a tuple in the form of:

(exitstatus, output)

If the command takes longer than timeout_duration seconds, it will be auto-killed and the following will be returned:

(255, _("ERROR: Timeout running shell command"))

utils.json_encode(obj)

On some platforms (CentOS 6.2, specifically) tornado.escape.json_decode doesn't seem to work just right when it comes to returning unicode strings. This is just a wrapper that ensures that the returned string is unicode.

utils.get_translation()

Looks inside GATEONE_DIR/server.conf to determine the configured locale and returns a matching locale.get_translation function. Meant to be used like this:

>>> from utils import get_translation
>>> _ = get_translation()

utils.gen_self_signed_ssl(path=None)

Generates a self-signed SSL certificate using pyOpenSSL or the openssl command depending on what's available, The resulting key/certificate will use the RSA algorithm at 4096 bits.

utils.gen_self_signed_openssl(path=None)

This method will generate a secure self-signed SSL key/certificate pair (using the openssl command) saving the result as 'certificate.pem' and 'keyfile.pem' to path. If path is not given the result will be saved in the current working directory. The certificate will be valid for 10 years.

utils.gen_self_signed_pyopenssl(notAfter=None, path=None)

This method will generate a secure self-signed SSL key/certificate pair (using pyOpenSSL) saving the result as 'certificate.pem' and 'keyfile.pem' in path. If path is not given the result will be saved in the current working directory. By default the certificate will be valid for 10 years but this can be overridden by passing a valid timestamp via the notAfter argument.

Examples:

>>> gen_self_signed_ssl(60 * 60 * 24 * 365) # 1-year certificate
>>> gen_self_signed_ssl() # 10-year certificate

utils.none_fix(val)

If val is a string that utlimately means 'none', return None. Otherwise return val as-is. Examples:

>>> none_fix('none')
None
>>> none_fix('0')
None
>>> none_fix('whatever')
'whatever'

utils.str2bool(val)

Converts strings like, 'false', 'true', '0', and '1' into their boolean equivalents. If no logical match is found, return False. Examples:

>>> str2bool('false')
False
>>> str2bool('1')
True
>>> st2bool('whatever')
False

utils.generate_session_id()

Returns a random, 45-character session ID. Example:

>>> generate_session_id()
"NzY4YzFmNDdhMTM1NDg3Y2FkZmZkMWJmYjYzNjBjM2Y5O"
>>>

utils.mkdir_p(path)

Pythonic version of "mkdir -p". Example equivalents:

>>> import commands
>>> mkdir_p('/tmp/test/testing') # Does the same thing as below:
>>> commands.getstatusoutput('mkdir -p /tmp/test/testing')

Note

This doesn't actually call any external commands.

utils.cmd_var_swap(cmd, session=None, session_hash=None, user_dir=None, user=None, time=None)

Returns cmd with special inline variables swapped out for their respective argument values. The special variables are as follows:

%SESSION%	session
%SESSION_HASH%	session_hash
%USERDIR%	user_dir
%USER%	user
%TIME%	time

This allows for unique or user-specific values to be swapped into command line arguments like so:

ssh_connect.py -M -S '/tmp/gateone/%SESSION%/%r@%L:%p'

The values passed into this function can be whatever you like. They don't necessarily have to match their intended values.

utils.short_hash(to_shorten)

Converts to_shorten into a really short hash depenendent on the length of to_shorten. The result will be safe for use as a file name.

utils.get_process_tree(parent_pid)

Returns a list of child pids that were spawned from parent_pid.

Note

Will include parent_pid in the output list.

utils.kill_dtached_proc_macos(session, term)

A Mac OS-specific implementation of kill_dtached_proc since Macs don't have /proc. Seems simpler than kill_dtached_proc() but actually having to call a subprocess is less efficient (due to the sophisticated signal handling required by shell_command()).

utils.killall_macos(session_dir)

A Mac OS X-specific version of killall since Macs don't have /proc.

utils.create_plugin_links(static_dir, templates_dir, plugin_dir)

Creates symbolic links for all plugins in the ./static/ and ./templates/ directories. The equivalent of:

root@host:~ $ ln -s *plugin_dir*/<plugin>/static *static_dir*/<plugin>
root@host:~ $ ln -s *plugin_dir*/<plugin>/templates *templates_dir*/<plugin>

This is so plugins can reference files in these directories using the following straightforward paths:

https://<gate one>/static/<plugin name>/<some file>
https://<gate one>/render/<plugin name>/<some file>

This function will also remove any dead links if a plugin is removed.

utils.get_plugins(plugin_dir)

Adds plugins' Python files to sys.path and returns a dictionary of JavaScript, CSS, and Python files contained in plugin_dir like so:

{
    'js': [ // NOTE: These would be be inside *plugin_dir*/static
        '/static/happy_plugin/whatever.js',
        '/static/ssh/ssh.js',
    ],
    'css': ['/cssrender?plugin=bookmarks&template=bookmarks.css'],
    // NOTE: CSS URLs will require '&container=<container>' and '&prefix=<prefix>' to load.
    'py': [ // NOTE: These will get added to sys.path
        'happy_plugin',
        'ssh'
    ],
}

*.js files inside of plugin_dir/<the plugin>/static will get automatically added to Gate One's index.html like so:

{% for jsplugin in jsplugins %}
    <script type="text/javascript" src="{{jsplugin}}"></script>
{% end %}

*.css files will get imported automatically by GateOne.init()

utils.load_plugins(plugins)

Given a list of plugins, imports them.

Note

Assumes they're all in sys.path.

utils.merge_handlers(handlers)

Takes a list of Tornado handlers like this:

[
    (r"/", MainHandler),
    (r"/ws", TerminalWebSocket),
    (r"/auth", AuthHandler),
    (r"/style", StyleHandler),
        ...
    (r"/style", SomePluginHandler),
]

...and returns a list with duplicate handlers removed; giving precedence to handlers with higher indexes. This allows plugins to override Gate One's default handlers. Given the above, this is what would be returned:

[
    (r"/", MainHandler),
    (r"/ws", TerminalWebSocket),
    (r"/auth", AuthHandler),
        ...
    (r"/style", SomePluginHandler),
]

This example would replace the default "/style" handler with SomePluginHandler; overriding Gate One's default StyleHandler.

utils.convert_to_timedelta(time_val)

Given a time_val (string) such as '5d', returns a datetime.timedelta object representing the given value (e.g. timedelta(days=5)). Accepts the following '<num><char>' formats:

Character	Meaning	Example
s	Seconds	'60s' -> 60 Seconds
m	Minutes	'5m' -> 5 Minutes
h	Hours	'24h' -> 24 Hours
d	Days	'7d' -> 7 Days

Examples:

>>> convert_to_timedelta('7d')
datetime.timedelta(7)
>>> convert_to_timedelta('24h')
datetime.timedelta(1)
>>> convert_to_timedelta('60m')
datetime.timedelta(0, 3600)
>>> convert_to_timedelta('120s')
datetime.timedelta(0, 120)

utils.convert_to_bytes(size_val)

Given a size_val (string) such as '100M', returns an integer representing an equivalent amount of bytes. Accepts the following '<num><char>' formats:

Character	Meaning	Example
B (or none)	Bytes	'100' or '100b' -> 100
K	Kilobytes	'1k' -> 1024
M	Megabytes	'1m' -> 1048576
G	Gigabytes	'1g' -> 1073741824
T	Terabytes	'1t' -> 1099511627776
P	Petabytes	'1p' -> 1125899906842624
E	Exabytes	'1e' -> 1152921504606846976
Z	Zettabytes	'1z' -> 1180591620717411303424L
Y	Yottabytes	'7y' -> 1208925819614629174706176L

Note

If no character is given the size_val will be assumed to be in bytes.

Tip

All characters will be converted to upper case before conversion (case-insensitive).

Examples:

>>> convert_to_bytes('2M')
2097152
>>> convert_to_bytes('2g')
2147483648

utils.process_opt_esc_sequence(chars)

Parse the chars passed from terminal.Terminal by way of the special, optional escape sequence handler (e.g. '<plugin>|<text>') into a tuple of (<plugin name>, <text>). Here's an example:

>>> process_opt_esc_sequence('ssh|user@host:22')
('ssh', 'user@host:22')

utils.raw(text, replacement_dict=None)

Returns text as a string with special characters replaced by visible equivalents using replacement_dict. If replacement_dict is None or False the global REPLACEMENT_DICT will be used. Example:

>>> test = '\x1b]0;Some xterm title'
>>> print(raw(test))
'^[]0;Some title^G'

utils.string_to_syslog_facility(facility)

Given a string (facility) such as, "daemon" returns the numeric syslog.LOG_* equivalent.

utils.create_data_uri(filepath)

Given a file at filepath, return that file as a data URI.

Raises a MimeTypeFail exception if the mimetype could not be guessed.

utils.human_readable_bytes(nbytes)

Returns nbytes as a human-readable string in a similar fashion to how it would be displayed by 'ls -lh' or 'df -h'.

utils.which(binary, path=None)

Returns the full path of binary (string) just like the 'which' command. Optionally, a path (colon-delimited string) may be given to use instead of os.environ ['PATH'].

utils.timeout_func(func, args=(), kwargs={}, timeout_duration=10, default=None)

Sets a timeout on the given function, passing it the given args, kwargs, and a default value to return in the event of a timeout. If default is a function that function will be called in the event of a timeout.

utils.valid_hostname(hostname, allow_underscore=False)

Returns True if the given hostname is valid according to RFC rules. Works with Internationalized Domain Names (IDN) and optionally, hostnames with an underscore (if allow_underscore is True).

The rules for hostnames:

• Must be less than 255 characters.
• Individual labels (separated by dots) must be <= 63 characters.
• Only the ASCII alphabet (A-Z) is allowed along with dashes (-) and dots (.).
• May not start with a dash or a dot.
• May not end with a dash.
• If an IDN, when converted to Punycode it must comply with the above.

IP addresses will be validated according to their well-known specifications.

Examples:

>>> valid_hostname('foo.bar.com.') # Standard FQDN
True
>>> valid_hostname('2foo') # Short hostname
True
>>> valid_hostname('-2foo') # No good:  Starts with a dash
False
>>> valid_hostname('host_a') # No good: Can't have underscore
False
>>> valid_hostname('host_a', allow_underscore=True) # Now it'll validate
True
>>> valid_hostname(u'ジェーピーニック.jp') # Example valid IDN
True

utils.recursive_chown(path, uid, gid)

Emulates 'chown -R uid:gid path' in pure Python

utils.drop_privileges(uid='nobody', gid='nogroup', supl_groups=None)

Drop privileges by changing the current process owner/group to uid/gid (both may be an integer or a string). If supl_groups (list) is given the process will be assigned those values as its effective supplemental groups. If supl_groups is None it will default to using 'tty' as the only supplemental group. Example:

drop_privileges('gateone', 'gateone', ['tty'])

This would change the current process owner to gateone/gateone with 'tty' as its only supplemental group.

Note

On most Unix systems users must belong to the 'tty' group to create new controlling TTYs which is necessary for 'pty.fork()' to work.

Tip

If you get errors like, "OSError: out of pty devices" it likely means that your OS uses something other than 'tty' as the group owner of the devpts filesystem. 'mount | grep pts' will tell you the owner.

utils.kill_dtached_proc(session, term)

Kills the dtach processes associated with the given term and all its sub-processes. Requires session so it can figure out the right processess to kill.

utils.killall(session_dir)

Kills all running Gate One terminal processes including any detached dtach sessions.

Session_dir :The path to Gate One's session directory.