Fixed some bugs found in GEN_FILE

genfile commands would not terminate properly or
would end up de-synced with the host in one way or
another.

to fix this, i updated the GEN_FILE data type and
sub-protocol to now define the commands as download
or upload on the NEW_CMD frame so clients can now
define the direction of the GEN_FILE data of the
various GEN_FILE commands at the very start instead
of trying to determine that at run time.

also fixed this by creating the onTerminate() virtual
function in CmdObject and have it call this function
when term() is called. this makes it possible to
properly put the command object in a reset state if
using parameters outside of the base class when
term() is called.

updated all documentation related to the GEN_FILE
sub-protocol to reflect these changes.

fixed ASYNC_DEBUG_TEXT to self correct the ipc type
to PRIV_IPC so no debug messages can accidentally be
sent to peers.
This commit is contained in:
Maurice ONeal 2019-12-14 13:12:25 -05:00
parent f6ea7239a0
commit 87d9eb0207
14 changed files with 135 additions and 117 deletions

View File

@ -47,7 +47,9 @@ format: ```[UTF-16LE_string] (no BOM)```
```GEN_FILE```
This is a file transfer type id that can be used to transfer any file type (music, photos, documents, etc...). It operates in its own protocol of sorts. The 1st GEN_FILE frame received by the host or client is TEXT parameters similar to what you see in terminal command lines with at least one of the arguments listed below. The next set of GEN_FILE frames received by the host or client is then the binary data that needs to be written to an open file or streamed until the limit defined in -len is meet.
The host or the client can be set as the sender or receiver of the GEN_FILE binary data. Which ever is designated as the receiver by the TEXT parameters need to send an empty GEN_FILE frame to start the process. An example of this can be found in section 3.3.
The host or the client can be set as the sender or receiver of the GEN_FILE binary data. This designation is determined by what the command defined in genfile type when the NEW_CMD frame was sent. A genfile type of 2 sets the client as the sender and the host as the receiver. Genfile type 3 sets the client as the receiver and the host as the sender.
see section 3.3 for an example of how GEN_FILE works.
arguments:
@ -61,10 +63,6 @@ arguments:
* **-single_step** | the presents of this argument tells both the client and host to operate in single step mode. single step mode causes the receiver of the binary data whether host or client to send an empty GEN_FILE frame after successfully receiving the data. this then tells the sender to send the next GEN_FILE frame containing binary data for the file and the cycle continues until len is meet. if this argument is not found, the sender can simply send all GEN_FILE data without waiting for an empty GEN_FILE from the receiver.
* **-to_host** | this argument should only come from the host and it will define the client as the sender and the host as the receiver.
* **-from_host** | opposite affect to *-to_host*. it defines the host as the sender and the client as the receiver.
* **-truncate** | this indicates to whoever is the receiver to truncate the file being written to.
* **-force** | in some cases, the receiver might need to overwrite the target file. the presents of this argument tells it to overwrite without asking the user. the host should never send this argument and the client should ignore it if it is received from the host.
@ -133,6 +131,7 @@ This is a data structure that carries information about a file system object (fi
notes:
1. 16bit terminated UTF-16LE strings are basically
terminated by 2 bytes of 0x00.
2. the symmlink target is empty if not a symmlink but
the terminator should still be present.
@ -161,9 +160,11 @@ This carry some user account and session information about a peer client connect
notes:
1. the session id is unique to the peer's session connection only. it
can change upon reconnection.
2. the user id is unique to the peer's user account. is stays constant
even when the user name changes and across all clients logged into
the same account.
3. the display name is the preffered display name of the peer. clients
are encouraged to use this rather than the user name when displaying
peer info to the user. if empty, it's ok to just fall back to the user
@ -189,7 +190,7 @@ This contains information about a new command that was added to the current sess
```
format:
1. bytes[0-1] 2bytes - 16bit LE unsigned int (command id)
2. bytes[2] 1byte - bool (0x01 or 0x00) (handles gen file)
2. bytes[2] 1byte - 8bit LE unsigned int (genfile type)
3. bytes[3-130] 128bytes - command name (TEXT - padded with 0x00)
4. bytes[131-258] 128bytes - library name (TEXT - padded with 0x00)
5. bytes[259-n] variable - short text (16bit null terminated)
@ -197,10 +198,13 @@ This contains information about a new command that was added to the current sess
7. bytes[n-n] variable - long text (16bit null terminated)
notes:
1. the handles gen file flag is a single byte 0x01 to indicate true and
0x00 to indicate false. clients need to be aware of which command
handles the GEN_FILE mini protocol because it requires user input at
both ends (host and client).
1. the genfile type is numerical value of 2, 3 or 0. a value of 2
indicates that the command handles/understands the GEN_FILE mini
protocol and it can be used to upload a file or other data to the
host. a value of 3 indicates the commmand downloads a file or other
data from the host. 0 simply indicates that the command doesn't use
or understand GEN_FILE.
2. the library name can contain the module name and/or extra informaion
the client can use to identify the library the command is a part of.
```
@ -233,6 +237,7 @@ This contain status information of a peer client when the peer changes sub-chann
1. if (is disconnected) is set true (0x01) the session id will no longer
be valid for that peer client so you should not make anymore attempts
to send data to it.
2. channel-sub ids is a string of 9byte channel-sub id combinations at
a fixed length of 54bytes (padded with 0x00). this indicates what
channels-subs the peer currently have open if the peer's channel ids
@ -272,8 +277,10 @@ This contains public information about a channel member.
notes:
1. a 16bit null terminated TEXT formatted string ended with 2 bytes of
(0x00) to indicate the end of the string data.
2. the member's privilege level can be any of the values discribed in
section [4.3](host_features.md).
3. is invite? indicates if this user has received an invite to join
that channel by has not accepted yet. if, accepted the user will
become a full member of the channel at the level indicated by this
@ -284,17 +291,19 @@ This contains public information about a channel member.
Setup:
* The host has a command called *upload_file* with a command id of *768* and handles the ```GEN_FILE``` data type.
* The host has a command called *upload_file* with a command id of *768* and handles the ```GEN_FILE``` data type with a genfile type of 2.
* The client has a file called */home/foo/bar.mp3* and wants to upload it to the host file */home/host/music/bar.mp3* and the client knows the file size is 512bytes.
Process:
To upload the file, the client calls command id *768* with the following text arguments (must still be sent as a GEN_FILE):
```-client_file "/home/foo/bar.mp3" -remote_file "/home/host/music/bar.mp3" -len 512```
The host will then return the following the text arguments to the client (also sent as a GEN_FILE):
```-to_host```
The host can then return text arguments to the client like (also sent as a GEN_FILE):
```-truncate and/or -single_step```
This argument from the host designates it as the receiver so it will be up to the host to send an empty ```GEN_FILE``` to indicate to the client that it was ready to start receiving binary data from the client to write to */home/host/music/bar.mp3*. If that file already exists, the host will need to ask the user to overwrite or not.
This only needs to be done if the command call needs to be modified by host and if the client supports such a thing. In this example, host will just return an empty GEN_FILE because there is no need for modification.
If the host indicates that it's ready for the upload, the client can then simply read 512 bytes from */home/foo/bar.mp3* and send the read bytes to the host command id *768* as a ```GEN_FILE```.
At this point, the host will then need to check of the destination file: /home/host/music/bar.mp3 already exists. If it does, the host will need to ask the user if it's ok to overwrite.
The host will then write the bytes received from the client to */home/host/music/bar.mp3* and then auto terminate the command since 512 bytes has been meet.
Once the host confirms it is ok to write to the destination file, it will then need to send another empty GEN_FILE to the client to confirm that it is ready to start receiving GEN_FILE frames from the client through command id *768* that will contain binary data to be written to the destination file until -len (512 bytes) is meet.

View File

@ -5,7 +5,7 @@ installer_file="$2"
src_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
bin_name="mrci"
app_version="1.1.2"
app_version="2.1.2"
app_name="MRCI"
install_dir="/opt/$bin_name"
bin_dir="/usr/bin"

View File

@ -116,6 +116,7 @@ void CmdObject::term()
{
flags = 0;
onTerminate();
postProc();
}
}

View File

@ -77,6 +77,7 @@ protected:
QString libName();
virtual void procIn(const QByteArray &, quint8) {}
virtual void onTerminate() {}
protected slots:

View File

@ -616,6 +616,11 @@ void CmdProcess::onDataFromProc(quint8 typeId, const QByteArray &data)
QString errMsg;
QTextStream errTxt(&errMsg);
if (async == ASYNC_DEBUG_TEXT)
{
typeId = PRIV_IPC;
}
if (validAsync(async, payload, errTxt))
{
if (typeId == PRIV_IPC)

View File

@ -366,7 +366,7 @@ void SetEmailTemplate::procIn(const QByteArray &binIn, quint8 dType)
proc();
}
}
else if (dType == GEN_FILE)
else if ((dType == GEN_FILE) || (dType == TEXT))
{
QStringList args = parseArgs(binIn, 9);
@ -427,7 +427,7 @@ void SetEmailTemplate::procIn(const QByteArray &binIn, quint8 dType)
flags |= MORE_INPUT;
emit procOut(toTEXT("-to_host"), GEN_FILE);
emit procOut(QByteArray(), GEN_FILE);
emit procOut(QByteArray(), GEN_FILE);
}
else

View File

@ -63,8 +63,8 @@ void mkPathForFile(const QString &path)
mkPath(QFileInfo(path).absolutePath());
}
DownloadFile::DownloadFile(QObject *parent) : CmdObject(parent) {file = new QFile(this);}
UploadFile::UploadFile(QObject *parent) : CmdObject(parent) {file = new QFile(this);}
DownloadFile::DownloadFile(QObject *parent) : CmdObject(parent) {file = new QFile(this); onTerminate();}
UploadFile::UploadFile(QObject *parent) : CmdObject(parent) {file = new QFile(this); onTerminate();}
Delete::Delete(QObject *parent) : CmdObject(parent) {}
Copy::Copy(QObject *parent) : CmdObject(parent) {src = new QFile(this); dst = new QFile(this);}
Move::Move(QObject *parent) : Copy(parent) {}
@ -85,11 +85,12 @@ QString FileInfo::cmdName() {return "fs_info";}
QString ChangeDir::cmdName() {return "fs_cd";}
QString Tree::cmdName() {return "fs_tree";}
void DownloadFile::clear()
void DownloadFile::onTerminate()
{
file->close();
buffSize = static_cast<qint64>(qPow(2, MAX_FRAME_BITS) - 1);
ssMode = false;
paramsSet = false;
dataSent = 0;
len = 0;
flags = 0;
@ -97,32 +98,34 @@ void DownloadFile::clear()
void DownloadFile::sendChunk()
{
if (buffSize > len) buffSize = len;
QByteArray data = file->read(buffSize);
QByteArray data = file->read(LOCAL_BUFFSIZE);
dataSent += data.size();
emit procOut(data, GEN_FILE);
mainTxt(QString::number(dataSent) + "/" + QString::number(len) + "\n");
mainTxt(QString::number(dataSent) + " / " + QString::number(len) + "\n");
if ((dataSent >= len) || file->atEnd())
{
clear();
onTerminate();
}
}
void DownloadFile::procIn(const QByteArray &binIn, quint8 dType)
{
if ((dType == GEN_FILE) && (flags & (MORE_INPUT | LOOPING)))
if ((dType == GEN_FILE) && binIn.isEmpty() && ssMode && paramsSet)
{
sendChunk();
}
else if (paramsSet)
{
flags |= LOOPING;
sendChunk();
}
else if (dType == GEN_FILE)
{
clear();
QStringList args = parseArgs(binIn, 11);
QString path = getParam("-remote_file", args);
QString offStr = getParam("-offset", args);
@ -142,53 +145,48 @@ void DownloadFile::procIn(const QByteArray &binIn, quint8 dType)
{
errTxt("err: Offset '" + offStr + "' is not a valid integer.\n");
}
else if (!isInt(lenStr))
else if (!lenStr.isEmpty() && !isInt(lenStr))
{
errTxt("err: Len '" + lenStr + "' is not valid integer.\n");
errTxt("err: Len '" + lenStr + "' is not a valid integer.\n");
}
else if (!QFileInfo(path).isFile())
{
errTxt("err: The remote file is not a file at all.\n");
errTxt("err: The remote file is not a file or does not exists.\n");
}
else if (!file->open(QFile::ReadOnly))
{
errTxt("err: Unable to open remote file for reading. reason: " + file->errorString() + "\n");
errTxt("err: Unable to open the remote file for reading. reason: " + file->errorString() + "\n");
}
else
{
QString genfileRet = "-from_host";
len = lenStr.toLongLong();
ssMode = argExists("-single_step", args);
paramsSet = true;
flags |= MORE_INPUT;
if ((len == 0) || (len > file->size()))
{
genfileRet.append(" -len " + QString::number(len));
len = file->size();
}
if (!argExists("-single_step", args))
{
flags |= LOOPING;
}
file->seek(offStr.toLongLong());
emit procOut(toTEXT(genfileRet), GEN_FILE);
emit procOut(toTEXT("-len " + QString::number(len)), GEN_FILE);
}
}
}
void UploadFile::clear()
void UploadFile::onTerminate()
{
file->close();
force = false;
confirm = false;
ssMode = false;
dataReceived = 0;
len = 0;
flags = 0;
offs = 0;
mode = nullptr;
}
@ -198,13 +196,13 @@ void UploadFile::wrToFile(const QByteArray &data)
file->write(data);
mainTxt(QString::number(dataReceived) + "/" + QString::number(len) + "\n");
mainTxt(QString::number(dataReceived) + " / " + QString::number(len) + "\n");
if (dataReceived >= len)
{
clear();
onTerminate();
}
else if (flags & SINGLE_STEP_MODE)
else if (ssMode)
{
emit procOut(QByteArray(), GEN_FILE);
}
@ -221,12 +219,14 @@ void UploadFile::run()
{
if (file->open(mode))
{
file->seek(offs);
emit procOut(QByteArray(), GEN_FILE);
}
else
{
errTxt("err: Unable to open the remote file for writing. reason: " + file->errorString() + "\n");
clear();
onTerminate();
}
}
@ -244,7 +244,7 @@ void UploadFile::procIn(const QByteArray &binIn, quint8 dType)
}
else if (noCaseMatch("n", ans))
{
clear();
onTerminate();
}
else
{
@ -257,29 +257,10 @@ void UploadFile::procIn(const QByteArray &binIn, quint8 dType)
}
else if (dType == GEN_FILE)
{
clear();
QStringList args = parseArgs(binIn, 11);
QString lenStr = getParam("-len", args);
QString offStr = getParam("-offset", args);
QString dst = getParam("-remote_file", args);
bool exists = QFileInfo(dst).exists();
file->setFileName(dst);
if (argExists("-truncate", args))
{
mode = QFile::ReadWrite | QFile::Truncate;
}
else
{
mode = QFile::ReadWrite;
}
if (argExists("-single_step", args))
{
flags |= SINGLE_STEP_MODE;
}
if (dst.isEmpty())
{
@ -299,16 +280,33 @@ void UploadFile::procIn(const QByteArray &binIn, quint8 dType)
}
else
{
if (argExists("-truncate", args))
{
mode = QFile::ReadWrite | QFile::Truncate;
}
else
{
mode = QFile::ReadWrite;
}
force = argExists("-force", args);
ssMode = argExists("-single_step", args);
len = lenStr.toLongLong();
offs = offStr.toLongLong();
flags |= MORE_INPUT;
file->seek(offStr.toLongLong());
file->setFileName(dst);
emit procOut(toTEXT("-to_host"), GEN_FILE);
emit procOut(QByteArray(), GEN_FILE);
if (exists && !force) ask();
else run();
if (QFileInfo(dst).exists() && !force)
{
ask();
}
else
{
run();
}
}
}
}
@ -396,7 +394,7 @@ void Delete::procIn(const QByteArray &binIn, uchar dType)
}
}
void Copy::clear()
void Copy::onTerminate()
{
fromQueue = false;
procedAFile = false;
@ -473,7 +471,7 @@ void Copy::run()
else
{
errTxt("err: Unable to re-create the source symlink at the destination path. writing to the path is not possible/denied.\n");
clear();
onTerminate();
}
}
else if (QFileInfo(srcPath).isDir())
@ -488,12 +486,12 @@ void Copy::run()
if (!dst->open(QFile::WriteOnly | QFile::Truncate))
{
errTxt("err: Unable to open the destination file '" + dstPath + "' for writing. reason: " + dst->errorString() + "\n");
clear();
onTerminate();
}
else if (!src->open(QFile::ReadOnly))
{
errTxt("err: Unable to open the source file '" + srcPath + "' for reading. reason: " + src->errorString() + "\n");
clear();
onTerminate();
}
else
{
@ -553,7 +551,7 @@ void Copy::procIn(const QByteArray &binIn, uchar dType)
else
{
preFinish();
clear();
onTerminate();
}
}
else if ((dType == TEXT) && (flags & MORE_INPUT))
@ -578,7 +576,7 @@ void Copy::procIn(const QByteArray &binIn, uchar dType)
}
else
{
clear();
onTerminate();
}
}
else if (fromQueue)
@ -611,7 +609,7 @@ void Copy::procIn(const QByteArray &binIn, uchar dType)
}
else if (dType == TEXT)
{
clear();
onTerminate();
QStringList args = parseArgs(binIn, 5);
bool force = argExists("-force", args);
@ -668,7 +666,7 @@ void Move::runOnMatchingVolume()
if (!QFile::rename(srcPath, dstPath))
{
errTxt("err: Unable to do move operation. it's likely the command failed to remove the existing destination object or writing to the path is not possible/denied.\n");
clear();
onTerminate();
}
}
@ -868,7 +866,7 @@ void ChangeDir::procIn(const QByteArray &binIn, quint8 dType)
}
}
void Tree::clear()
void Tree::onTerminate()
{
queue.clear();
@ -917,7 +915,7 @@ void Tree::printList(const QString &path)
if (queue.isEmpty())
{
clear();
onTerminate();
}
else
{

View File

@ -31,17 +31,18 @@ class DownloadFile : public CmdObject
private:
QFile *file;
qint64 buffSize;
qint64 len;
qint64 dataSent;
bool ssMode;
bool paramsSet;
void sendChunk();
void onTerminate();
public:
static QString cmdName();
void clear();
void procIn(const QByteArray &binIn, quint8 dType);
explicit DownloadFile(QObject *parent = nullptr);
@ -59,11 +60,13 @@ private:
QFile *file;
qint64 len;
qint64 dataReceived;
qint64 offs;
bool ssMode;
bool confirm;
bool force;
void wrToFile(const QByteArray &data);
void onTerminate();
void run();
void ask();
@ -71,7 +74,6 @@ public:
static QString cmdName();
void clear();
void procIn(const QByteArray &binIn, quint8 dType);
explicit UploadFile(QObject *parent = nullptr);
@ -109,6 +111,7 @@ protected:
void ask();
void run();
void onTerminate();
QFile *src;
QFile *dst;
@ -131,7 +134,6 @@ public:
static QString cmdName();
void clear();
void procIn(const QByteArray &binIn, quint8 dType);
explicit Copy(QObject *parent = nullptr);
@ -231,12 +233,12 @@ private:
bool noHidden;
void printList(const QString &path);
void onTerminate();
public:
static QString cmdName();
void clear();
void procIn(const QByteArray &binIn, quint8 dType);
explicit Tree(QObject *parent = nullptr);

View File

@ -33,10 +33,9 @@ ListDBG::ListDBG(QObject *parent) : TableViewer(parent)
addTableColumn(TABLE_DMESG, COLUMN_LOGENTRY);
}
ListCommands::ListCommands(const QStringList &cmdList, const QStringList &gen, QObject *parent) : CmdObject(parent)
ListCommands::ListCommands(const QStringList &cmdList, QObject *parent) : CmdObject(parent)
{
list = cmdList;
genfileList = gen;
}
HostInfo::HostInfo(QObject *parent) : CmdObject(parent) {}
@ -67,15 +66,23 @@ void ListCommands::onIPCConnected()
for (auto&& cmdName : list)
{
QByteArray frame;
QByteArray boolByte = QByteArray(1, 0x00);
QByteArray genType = QByteArray(1, 0x00);
if (genfileList.contains(cmdName))
if (cmdName == DownloadFile::cmdName())
{
boolByte = QByteArray(1, 0x01);
genType = QByteArray(1, GEN_DOWNLOAD);
}
else if (cmdName == UploadFile::cmdName())
{
genType = QByteArray(1, GEN_UPLOAD);
}
else if (cmdName == SetEmailTemplate::cmdName())
{
genType = QByteArray(1, GEN_UPLOAD);
}
frame.append(QByteArray(2, 0x00));
frame.append(boolByte);
frame.append(genType);
frame.append(fixedToTEXT(cmdName, BLKSIZE_CMD_NAME));
frame.append(fixedToTEXT(libName(), BLKSIZE_LIB_NAME));
frame.append(nullTermTEXT(shortText(cmdName)));

View File

@ -21,6 +21,8 @@
#include "../cmd_object.h"
#include "../shell.h"
#include "table_viewer.h"
#include "fs.h"
#include "acct_recovery.h"
class ListCommands : public CmdObject
{
@ -29,7 +31,6 @@ class ListCommands : public CmdObject
private:
QStringList list;
QStringList genfileList;
QString shortText(const QString &cmdName);
QString ioText(const QString &cmdName);
@ -41,7 +42,7 @@ private slots:
public:
explicit ListCommands(const QStringList &cmdList, const QStringList &gen, QObject *parent = nullptr);
explicit ListCommands(const QStringList &cmdList, QObject *parent = nullptr);
};
//--------------------------------------

View File

@ -201,6 +201,12 @@ enum TypeID : quint8
RESUME_CMD = 29
};
enum GenFileType : quint8
{
GEN_UPLOAD = 2,
GEN_DOWNLOAD = 3
};
enum ChannelMemberLevel : quint8
{
OWNER = 1,

View File

@ -37,7 +37,7 @@
#include "shell.h"
#define APP_NAME "MRCI"
#define APP_VER "2.0.2"
#define APP_VER "2.1.2"
#define APP_TARGET "mrci"
#ifdef Q_OS_WIN

View File

@ -142,17 +142,6 @@ QStringList Module::pubCmdList()
return ret;
}
QStringList Module::genfileList()
{
QStringList ret;
ret << DownloadFile::cmdName();
ret << UploadFile::cmdName();
ret << SetEmailTemplate::cmdName();
return ret;
}
QStringList Module::rankExemptList()
{
QStringList ret;
@ -279,7 +268,7 @@ bool Module::runCmd(const QString &name)
void Module::listCmds(const QStringList &list)
{
new ListCommands(list, genfileList(), this);
new ListCommands(list, this);
}
bool Module::start(const QStringList &args)

View File

@ -49,7 +49,6 @@ private:
QStringList userCmdList();
QStringList pubCmdList();
QStringList rankExemptList();
QStringList genfileList();
public: