drogulus.net

Networking layer for the Drogulus.

  • messages.py - internal representations of messages and functions needed to serialise them.
  • protocol.py - contains the low level networking code needed for communication between nodes on the network.
  • validators.py - functions used to validate messages received from other nodes on the network.

drogulus.net.messages

Contains classes that represent messages sent down the wire between nodes on the network. Also contains functions for encoding/decoding to/from MessagePack. Every message contains at least three mandatory fields: uuid (identifying the interaction), node (the ID of the sender of the message) and version (indicating the version of Drogulus the sender is running).

Named tuples are used because they are lightweight, immutable and indexable (thus mimicing the dict like structure of the msgpack encoded message passed down the wire). See http://bugs.python.org/issue9391 for Hettinger’s advice on adding docstrings to namedtuples and the reason why the classes are declared in the way that they are.

class drogulus.net.messages.Error[source]

Represents an error message to be sent to the calling node on the network.

  • uuid - the ID of the request that generated the error.
  • node - the ID of the node sending the message.
  • code - the code that identifies the specific error.
  • title - a description of the type of error generated.
  • details - diagnostic details expressed as a dict of string key/values.
  • version - the protocol version the message conforms to.
class drogulus.net.messages.FindNode[source]

A “find node” message requests k nodes from the other nodes on the network that are closest to the given key. The value k is the maximum number of nodes that can be stored in a k-bucket and is set in the constants module. The original Kademlia paper names this variable and recommends its value as 20.

  • uuid - the ID of the FindNode request (generated by the requestee).
  • node - the ID of the node sending the message.
  • key - the key in the DHT that is being targetted.
  • version - the protocol version the message conforms to.
class drogulus.net.messages.FindValue[source]

A “find value” message will cause the other node to return the corresponding value if the given key is in its store. Otherwise it returns k nodes that it knows about that are closest to the given key.

  • uuid - the ID of the FindValue request (generated by the requestee).
  • node - the ID of the node sending the message.
  • key - the key in the DHT whose value is being requested.
  • version - the protocol version the message conforms to.
class drogulus.net.messages.Nodes[source]

A response to either a FindNode or FindValue request that contains a list of nodes on the DHT that are close to the requested key.

  • uuid - the ID of the request that is causing the response.
  • node - the ID of the node sending the message.
  • nodes - a list of nodes on the DHT that are close to the requested key.
  • version - the protocol version the message conforms to.
class drogulus.net.messages.Ping[source]

A “ping” message is sent to another node on the network to determine if it is still contactable.

  • uuid - the ping’s request ID (a random value generated by the requestor

    that uniquely identifies the request).

  • node - the ID of the node sending the message.

  • version - the protocol version the message conforms to.

class drogulus.net.messages.Pong[source]

A “pong” message is sent as a confirmation response. This is usually the result of a ping request but may be used to confirm reciept of any other sort of request when no further data is expected to be returned.

  • uuid - the request ID of the source of the response.
  • node - the ID of the node sending the message.
  • version - the protocol version the message conforms to.
class drogulus.net.messages.Store[source]

A “store” message instructs another node on the network to store the given key/value pair.

  • uuid - the ID of the Store request (generated by the requestee).

  • node - the ID of the node sending the message.

  • key - the SHA512 value of the compound key used as the actual key on the

    DHT.

  • value - the value to be stored in the DHT.

  • timestamp - a timestamp indicating when this key/value pair was

    originally generated as an integer representing the time in seconds since the Epoch (so called POSIX time, see https://en.wikipedia.org/wiki/Unix_time).

  • expires - a timestamp indicating a point in time after which this

    key/value pair can be removed (expired) from the DHT. Expressed as an integer representing the time in seconds since the Epoch (so called POSIX time). If the value is less than or equal to zero then the key/value pair should never expire.

  • created_with - the version of the protocol used to generate the item.

  • public_key - the public key of the person storing the value.

  • name - the human-readable name of the key.

  • meta - a dictionary containing key/value strings for user defined

    metadata.

  • sig - the cryptographic signature for the value, timestamp, expires,

    name, meta and created_with fields.

  • version - the protocol version the message conforms to.

The provenance of the message is guaranteed through cryptography:

The ‘sig’ field is created with the private key of the person storing the key/value pair. It’s derived from the SHA512 hash of the SHA512 hashes of the ‘value’, ‘timestamp’, ‘expires’, ‘name’, ‘meta’ and ‘created_with’ fields. This mechanism ensures that the public_key used in the compound key is valid (i.e. it validates the sig field given the correct SHA512 hash) and also ensures that the ‘value’, ‘timestamp’, ‘expires’, ‘name’, ‘meta’ and ‘created_with’ fields have not been tampered with.

The ‘key’ value is a compound key. It is a SHA512 hash of the SHA512 hashes of the ‘public_key’ and ‘name’ fields. The ‘public_key’ and ‘name’ fields are used to ensure that the compound ‘key’ field is correct.

class drogulus.net.messages.Value[source]

A response to a FindValue request. Contains all the information known by the responder about the key/value pair. Such complete information can be used to check the provenance of the data as described in the documentation for the Store message described above.

  • uuid - the ID of the request that is causing the response.

  • node - the ID of the node sending the message.

  • key - the SHA512 value of the compound key used as the actual key on the

    DHT.

  • value - the value found in the DHT.

  • timestamp - a timestamp indicating when this key/value pair was

    originally generated as a floating point number representing the time in seconds since the Epoch (so called POSIX time, see https://en.wikipedia.org/wiki/Unix_time).

  • expires - a timestamp indicating a point in time after which this

    key/value pair can be removed (expired) from the DHT. Expressed as an integer representing the time in seconds since the Epoch (so called POSIX time). If the value is less than or equal to zero then the key/value pair should never expire.

  • created_with - the version of the protocol used to generate the item.

  • public_key - the public key of the person who stored the value.

  • name - the human-readable name of the key.

  • meta - a dictionary containing key/value strings for user defined

    metadata.

  • sig - the cryptographic signature for the value, timestamp, expires,

    name, meta and created_with fields.

  • version - the protocol version the message conforms to.

drogulus.net.messages.from_msgpack(raw)[source]

Returns an instance of the correct message class given the msgpack encoded data in the raw string. Encapsulates a variety of cleaning and checking of the raw message from the (potentially dangerous) external network.

drogulus.net.messages.make_message(klass, data)[source]

Returns an instance of the referenced namedtuple based class that is created from the raw data. Data will be validated and an exception raised if this fails.

drogulus.net.messages.to_msgpack(message)[source]

Returns a string representation of the message object encoded using msgpack.

drogulus.net.protocol

Contains a definition of the low-level networking protocol used by the Drogulus.

class drogulus.net.protocol.DHTFactory(node)[source]

DHT Factory class that uses the DHTProtocol.

protocol

alias of DHTProtocol

class drogulus.net.protocol.DHTProtocol[source]

The low level networking protocol.

Msgpack (http://msgpack.org/) encoded payloads are transported as netstrings (http://cr.yp.to/proto/netstrings.txt).

The payload is simply a dictionary of attributes. Please see the classes representing each type of request/response type for what these attributes represent.

To the external world messages come in, messages go out (and implementation details are hidden).

except_to_error(exception)[source]

Given a Python exception will return an appropriate Error message instance.

sendMessage(msg, loseConnection=False)[source]

Sends the referenced message to the connected peer on the network. If loseConnection is set to true the connection will be dropped once the message has been sent.

stringReceived(raw)[source]

Handles incoming requests by unpacking them and instantiating the correct request class before passing them to the Node instance for further processing. If the message cannot be unpacked or is invalid an appropriate error message is returned to the originating caller.

drogulus.net.validators

Contains functions that are used to validate the type and content of fields in messages sent between nodes in the DHT.

drogulus.net.validators.validate_code(val)[source]

Returns a boolean indication that an error code is valid.

drogulus.net.validators.validate_meta(val)[source]

Returns a boolean to indicate that a meta-data field is a dictionary of key / value strings.

drogulus.net.validators.validate_node(val)[source]

Returns a boolean to indicate if the passed in tuple conforms to a specification of another node within the DHT. A valid node is a tuple with four items:

  • A string representation of the SHA512 ID of the node.
  • A string representation of the node’s IP address.
  • An integer representation of the node’s port within a valid range of port values.
  • A string representation of the version of Drogulus the remote node conforms to.
drogulus.net.validators.validate_nodes(val)[source]

Returns a boolean to indicate that a field is a tuple that may contain information about nodes.

drogulus.net.validators.validate_string(val)[source]

Returns a boolean to indicate that a field is a string of some sort.

drogulus.net.validators.validate_timestamp(val)[source]

Returns a boolean indication that a field is a valid timestamp - a floating point number representing the time in seconds since the Epoch (so called POSIX time, see https://en.wikipedia.org/wiki/Unix_time).

drogulus.net.validators.validate_value(val)[source]

Returns a boolean to indicate that a value stored in the DHT is valid. Currently all values are valid although in the future, size may be limited.