Server Mode

In addition to running MassMine interactively on the command line, MassMine can be run as a background process using its server mode. In server mode, MassMine opens a TCP port and listens for commands on that socket. Commands are processed and the resulting data are written back to the same port. Most of MassMine’s functionality is provided through this API—with the exception of authentication tasks. Server mode can be started with the following command (the port number 4242 can be substituted with any other appropriate free port number):

massmine --server=4242

While server mode allows the user to manually submit command requests over sockets, server mode is intended for controlling MassMine with other software (e.g., calling MassMine from other programs/scripts, to allow the creation of a graphical user interface front-end, etc.). As such, the documentation provided below is intended for software developers.

Server Mode Behavior

Once the MassMine server is running, you can send commands over TCP to the specified port. MassMine’s behavior is:

  1. Listen on the specified port for JSON strings.
  2. Convert JSON strings into internal commands.
  3. Run the internal command.
  4. Write the results, in line-oriented JSON, back to the same specified port.
  5. Write the string “0” (indicating success) or “1” (indicating failure) to the specified port to signal the end of the data stream. If “1” is returned, the previous JSON array will contain an element called “error” which contains the error information.
  6. Repeat until the “stop” task is requested, at which time the massmine process itself will terminate (after closing ports, etc.).

Clients set to read from the massmine port should be prepared for potentially very long timeouts (e.g., 15 minutes) in the middle of a data transmission. This is because massmine will pause to comply with rate limits set by upstream data sources. Clients should check for the “end of data stream” signals by monitoring for the “0” or “1” signals issued in step 5 above. After the signal has been caught, additional commands can be issued to massmine.

Commands should be sent as JSON structures containing name:value pairs that match the standard command line arguments issued to massmine. For example, the equivalent to the following command request:

massmine --task=twitter-search --query=politics --count=10

Could be called in “server mode” by first starting massmine:

massmine --server=4242

and then passing the following JSON structure to port 4242:

{"task":"twitter-search","query":"politics","count":10}

Once the data have been successfully returned, the calling program can the issue another data request, or massmine can be closed with the special command:

{"task":"stop"}

Minimal Working Example

To test MassMine’s server mode, a quick solution entails the use of the netcat command line tool. netcat provides the means to easily read/write data over TCP sockets. It is often already installed or readily available in the package managers for most Unix-like operating systems. A simple test case follows: