Made some major changes to the project to facilitate a lighter code base and the must flexible module interface possible. -the mutli-process architecture now operate at the command object level so each command now operate in it's own process instead of a single process handling multiple command objects. -each module is now an independent application that will now tell the session object all of the commands it can run via named pipe. during command execution, it will run the requested command object also running io with the session object via named pipe. with this change, it is now possible for modules to be developed in different versions or QT or entirely different languages. the only requirement is the need to support named pipes. shared memory segments is also a nice to have but not needed. -clients can now run multiple instances of the same command via changes to the protocol. mrci frames will now include a branch id along with the command id. the branch id can be used by clients to differentiate the io between instances of the same command. -the command states are longer controlled by a single object. it will now be up to the command object (internal/exterenal) to send an IDLE frame to the client to notify it that the command has finished. the session object will still track if the command is in idle state or not but not directly control it. -must async commands now use binary formatted data instead of TEXT as a way to reduce overhead. -all command objects can now send async commands. it is no longer limited to just internal commands, however; the data of these async commands are verified by session in some way to prevent host crashing due to malformed data. -changed up the database structure to rely more on user ids, channel ids and removed all foreign keys pointing to user names, channel names and sub-channel names. also removed the groups table altogether. instead, the host rank is now directly attached to the user data in the users table. -changed the query object to now support the INNER JOIN SQL clause. this change was needed to support the new database structure. -version negotiation is now one-way via tcp connection or module interface. the host will make it's own verion numner known to the client connected via tcp or the module connected via named pipe. it will now be entirely up to the client or module to decide if they support the host. another change in this regard is the removal of the import rev for the modules. compatibility for modules shall now use just the host verion. -removed ls_cmds and cmd_info. the NEW_CMD frame now carries all information about the command (cmd_id, cmd_name, summery, io and full_description) so it is now possible for the clients to display the command documentation instead of the host. Documentation for the internal commands were updated to reflect the changes but all other documentation will need to be updated in the near future.
13 KiB
4.1 Type IDs
All mrci frames transferred throughout this application are usually passed into various functions using a uchar
numeric value to indicate the type of data being passed with the QByteArray
that are also usually passed into the functions. The type id enum values are as follows:
enum TypeID : quint8
{
GEN_FILE = 1,
TEXT = 2,
ERR = 3,
PRIV_TEXT = 4,
IDLE = 5,
HOST_CERT = 6,
FILE_INFO = 7,
PEER_INFO = 8,
MY_INFO = 9,
PEER_STAT = 10,
P2P_REQUEST = 11,
P2P_CLOSE = 12,
P2P_OPEN = 13,
BYTES = 14,
SESSION_ID = 15,
NEW_CMD = 16,
CMD_ID = 17,
BIG_TEXT = 18,
TERM_CMD = 19,
HOST_VER = 20,
PRIV_IPC = 21,
PUB_IPC = 22,
PUB_IPC_WITH_FEEDBACK = 23,
PING_PEERS = 24,
CH_MEMBER_INFO = 25,
CH_ID = 26
};
4.2 Type Descriptions
TEXT
This is text that can be displayed directly to the user or pass arguments for the command object to process.
format: [UTF-16LE_string] (no BOM)
GEN_FILE
This is a file transfer type id that can be used to transfer any type of file type (music, photos, documents, etc...). It operates in its own protocol of sorts. The 1st GEN_FILE frame received by the host or client is TEXT parameters similar to what you see in terminal command lines with at least one of the arguments listed below. The next set of GEN_FILE frames received by the host or client is then the binary data that needs to be written to an open file or streamed until the limit defined in -len is meet.
The host or the client can be set as the sender or receiver of the GEN_FILE binary data. Which ever is designated as the receiver by the TEXT parameters need to send an empty GEN_FILE frame to start the process. An example if this can be found in section 4.3.
arguments:
-
-len (int) | this is the integer value of the file size or amount of bytes to read/write.
-
-offset (int) | this integer position tells where in the source or destination file to start reading/writing.
-
-client_file (string) | this is the file path to the source/destination file in the client's file system.
-
-remote_file (string) | this is the file path to the source/destination file in the host file system.
-
-single_step | the presents of this argument tells both the client and host to operate in single step mode. single step mode causes the receiver of the binary data whether host or client to send an empty GEN_FILE frame after successfully receiving the data. this then tells the sender to send the next GEN_FILE frame containing binary data for the file and the cycle continues until len is meet. if this argument is not found, the sender can simply send all GEN_FILE data without waiting for an empty GEN_FILE from the receiver.
-
-to_host | this argument should only come from host and it will define the client as the sender and the host as the receiver.
-
-from_host | opposite affect to -to_host. it defines the host as the sender and the client as the receiver.
-
-truncate | this indicates to whoever is the receiver to truncate the file being written to.
-
-force | in some cases, the receiver might need to overwrite the target file. the presents of this argument tells it to overwrite without asking the user. the host should never send this argument and the client should ignore it if it is received from the host.
ERR
This type id is similar to TEXT except it indicates that this is an error message that can be displayed directly to the user if needed.
PRIV_TEXT
This id can be treated exactly like TEXT except this should tell the client to hide or do not echo the next TEXT data that the host is expecting, like a password or other sensitive text data.
BIG_TEXT
Also formatted exactly like TEXT but this indicates to the client that this is a large body of text that is recommended to be word wrapped when displaying to the user. It can contain line breaks so clients are also recommended to honor those line breaks.
IDLE
This doesn't carry any actual data, instead this indicates that the command id that sent it has finished it's task. Commands objects are not allowed to send this directly, the CmdExecutor
will handle it.
HOST_CERT
Just as the name implies, this data type is used by the host to send the host SSL certificate while setting up an SSL connection.
FILE_INFO
This is a data structure that carries information about a file system object (file,dir,link).
format:
1. bytes[0] - flags (8bit little endian uint)
2. bytes[1-8] - creation time in msec since Epoch UTC (64bit little endian uint)
3. bytes[9-16] - modification time in msec since Epoch UTC (64bit little endian uint)
4. bytes[17-24] - file size (64bit little endian uint)
5. bytes[25-variable] - file name (UTF16-LE string, 16bit terminated)
6. bytes[variable] - symmlink target if it is a symmlink (UTF16-LE string, 16bit terminated)
notes:
1. 16bit terminated UTF-16LE strings are basically
terminated by 2 bytes of 0x00.
2. the symmlink target is empty if not a symmlink but
the terminator should still be present.
flags:
1. bit 0 - true if the object is a file
2. bit 1 - true if the object is a directory
3. bit 2 - true if the object is a symmlink
4. bit 3 - true if the current user have read permissions
5. bit 4 - true if the current user have write permissions
6. bit 5 - true if the current user have execute permissions
7. bit 6 - true if the object exist in the file system. if symmlink,
this determines if the symm target exists or not.
PEER_INFO
This carry some user account and session information about a peer client connected to the host.
format:
1. bytes[0-27] 28bytes - session id (224bit hash)
2. bytes[28-59] 32bytes - user id (256bit hash)
3. bytes[60-107] 48bytes - user name (TEXT - padded with empty spaces)
4. bytes[108-235] 128bytes - app name (TEXT - padded with empty spaces)
5. bytes[236-299] 64bytes - disp name (TEXT - padded with empty spaces)
notes:
1. the session id is unique to the peer's session connection only. it
can change upon reconnection.
2. the user id is unique to the peer's user account. is stays constant
even when the user name changes and across all clients logged into
the same account.
3. the display name is the preffered display name of the peer. clients
are encouraged to use this rather than the user name when displaying
peer info to the user. if empty, it's ok to just fall back to the user
name.
MY_INFO
This contains all of the information found in PEER_INFO
for the local session but also includes the following:
format:
1. bytes[300-427] 128bytes - email (TEXT - padded with empty spaces)
2. bytes[428-431] 4bytes - host rank (32bit unsigned int)
3. bytes[432] 1byte - is email confirmed? (0x00 false, 0x01 true)
NEW_CMD
This contains information about a new command that was added to the current session.
format:
1. bytes[0-1] 2bytes - 16bit LE unsigned int (command id)
2. bytes[2] 1byte - bool (0x01 or 0x00) (handles gen file)
3. bytes[3-130] 128bytes - command name (TEXT - padded with 0x00)
4. bytes[131-258] 128bytes - library name (TEXT - padded with 0x00)
5. bytes[259-n] variable - short text (16bit null terminated)
6. bytes[n-n] variable - io text (16bit null terminated)
7. bytes[n-n] variable - long text (16bit null terminated)
notes:
1. the handles gen file flag is a single byte 0x01 to indicate true and
0x00 to indicate false. clients need to be aware of which command
handles the GEN_FILE mini protocol because it requires user input at
both ends (host and client).
2. the library name can contain the module name and/or extra informaion
the client can use to identify the library the command is a part of.
CMD_ID
This type id carries a 16bit unsigned LE int representing a command id.
format: 2bytes - 16bit LE unsigned int (command id)
PEER_STAT
This contain status information of a peer client when the peer changes sub-channels or disconnects from the host.
format:
1. bytes[0-27] 28bytes - session id (224bit hash)
2. bytes[28-81] 54bytes - channel-sub ids
3. bytes[82] 1byte - is disconnected? (0x00 false, 0x01 true)
notes:
1. if (is disconnected) is set true (0x01) the session id will no longer
be valid for that peer client so you should not make anymore attempts
to send data to it.
2. channel-sub ids is a string of 9byte channel-sub id combinations at
a fixed length of 54bytes (padded with 0x00). this indicates what
channels-subs the peer currently have open if the peer's channel ids
no longer match with your session, it can be considered inactive or
disconnected since you will no longer send/receive data with this peer.
SESSION_ID
This is a fixed length 28byte(224bit) sha3 hash of a client's session id connected to the host. This is unique to just the client's tcp connection with the host. This can change upon re-connection.
format: 28bytes - session id (224bit sha3 hash)
P2P_REQUEST
This is formatted extactly like PEER_INFO
except this is allowed to be sent directly to the target peer without restriction when using the ExternCommand::toPeer()
signal (Command_Objects.md, section 2.5). It will be up to the target peer to respond with a P2P_OPEN
for the host to then unrestrict toPeer()
so it will then be able to send/received other TypeIDs with this peer until P2P_CLOSE
is sent/received. P2P_CLOSE
can also be sent to decline the request. When sending this TypeID, the CmdExecutor
will ignore the data passed into toPeer()
from the command object and substitute with it's own generated P2P_REQUEST
to prevent fraudulent request from getting sent out to all sessions; because of this, it is ok and preferable to send an empty P2P_REQUEST
with the toPeer()
signal.
P2P_OPEN
This contains a 28byte session id hash of the peer session that you or the peer will allow direct communication with when using the ExternCommand::toPeer()
signal.
format: 28bytes - session id (224bit sha3 hash)
P2P_CLOSE
This contains a 28byte session id hash of the peer session that you or the peer want to close direct communication with when using the ExternCommand::toPeer()
signal.
format: 28bytes - session id (224bit sha3 hash)
BYTES
This contains arbitrary binary data of any format that is not specialized for any internal objects in the host.
CH_MEMBER_INFO
This contains public information about a channel member.
format:
1. bytes[0-7] 8bytes - channel id (64bit unsigned int)
2. bytes[8-39] 32bytes - user id (256bit hash)
3. bytes[40] 1byte - is invite? (0x00=false, 0x01=true)
4. bytes[41] 1byte - member's channel privilege level (8bit unsigned int)
5. bytes[42-n] variable - user name (TEXT - 16bit null terminated)
6. bytes[n-n] variable - display name (TEXT - 16bit null terminated)
7. bytes[n-n] variable - channel name (TEXT - 16bit null terminated)
notes:
1. a 16bit null terminated TEXT formatted string ended with 2 bytes of
(0x00) to indicate the end of the string data.
2. the member's privilege level can be any of the values discribed in
section [5.3](Host_Features.md).
3. is invite? indicates if this user has received an invite to join
that channel by has not accepted yet. if, accepted the user will
become a full member of the channel at the level indicated by this
data type.
4.3 GEN_FILE Example
Setup:
- The host has a command called upload_file with a command id of 768 and handles the
GEN_FILE
data type. - The client has a file called /home/foo/bar.mp3 and wants to upload it to the host file /home/host/music/bar.mp3 and the client knows the file size is 512bytes.
To upload the file, the client calls command id 768 with the following text arguments (must still be sent as a GEN_FILE):
-client_file "/home/foo/bar.mp3" -remote_file "/home/host/music/bar.mp3" -len 512
The host will then return the following the text arguments to the client (also sent as a GEN_FILE):
-to_host
This argument from the host designates it as the receiver so it will be up to the host to send an empty GEN_FILE
to indicate to the client that it was ready to start receiving binary data from the client to write to /home/host/music/bar.mp3. If that file already exists, the host will need to ask the user to overwrite or not.
If the host indicates that it's ready for the upload, the client can then simply read 512 bytes from /home/foo/bar.mp3 and send the read bytes to the host command id 768 as a GEN_FILE
.
The host will then write the bytes received from the client to /home/host/music/bar.mp3 and then auto terminate the command since 512 bytes has been meet.