Add plugins support

[glassez]

This adds an experimental feature to support Lua plugins.
This is not a replacement/alternative to Search Plugins and has nothing to do with them. These are general purpose plugins.

Motivation

The idea of adding plugin support has been on my mind for a very long time, but I still haven't had time to take it seriously. It consists in providing users with the opportunity to integrate into their qBittorrent some additional functions that are useful to them, which are unlikely to be added to the core of qBittorrent (for example, due to the reluctance to overload it with features that have a very limited user audience. Users occasionally request some features that seem to be relevant only to themselves, so we usually reject them).
We also have some functions in qBittorrent that have limited integration with the main code, so they can easily be implemented by plugins, and can later be removed from the core. I mean, something like sending email notifications and launching external programs. These are all functions that do not have deep integration with the main logic of the application, they just obtain some data and transmit it to the outside world. In addition, they are unlikely to be used by most users. But at the same time, some individuals request their extension from time to time, for example, by executing at other events. I consider it highly undesirable to overload the core with such functions.
Note that it is not initially intended to provide any official plugins (except as plugin examples). However, this is quite possible in the future, if anyone wants to maintain them.

Interface

The main interface of plug-ins with the application core is "event handling". Event handlers are called for all installed plugins (provided, of course, that the plugin has a corresponding handler defined). At the same time, plug-ins (deliberately) live permanently during the lifetime of the application (well, either until they are uninstalled or updated), so they can store some data and track their changes.
Plugins can also be "user invocable" by providing the invoke() function. In this case, they can also be invoked by the user (for example, through the Plugins menu).

Implementation notes

The feature is based on the core Lua library (of the version 5.5 ATTOW) and LuaBridge3.
I deliberately chose to integrate their source code into the qBittorrent source tree in order to facilitate its building and distribution on different target platforms, and this should ensure more controlled compatibility with the plugins code, since we will always have a specific version of Lua and update it, if necessary, only in major qBittorrent updates.

Current state

Currently, all management functions are ready, such as installing, updating, and removing plug-ins, as well as enabling/disabling them. They are supported by GUI.
I have added support for most torrent-related events and their basic properties, so the rest can be added incrementally later.
"Invocable" plugins are listed under "Plugins" menu and can be invoked from there.
I have added several examples of plugins that demonstrate their main features and can serve as a basis for developing custom plug-ins.

Note for reviewers

Please refrain from suggesting any improvements that are not mandatory at the initial stage. Quite a lot of work has already been done to provide this PR, so I would prefer to focus on fixing the obvious bugs and merging it into the qBittorrent code. This would make it easier for me to maintain and improve it further.

[Chocobo1]

I'm not familiar with lua and this new plugin system. Here are some high level questions before taking a closer look.

• Where does the lua plugin live? A new process or another thread within qbt?
• When the lua plugin is badly programmed:
  • Can it crash the qbt process?
  • Can it hang the qbt process with some forever loop and never return?
  • Is it able to manipulate qbt memory space directly via pointer?

[glassez]

• Where does the lua plugin live? A new process or another thread within qbt?

No. This is a truly embedded engine, unlike the "crutches" with the launch of external programs as plugins, as we have used in the Search Engine subsystem. Plugins are executed in the qBittorrent process. And they are executed in the main thread, since the qBittorrent core is not multithreaded. Of course, this has certain disadvantages, but they are due to its advantages.

• Can it crash the qbt process?

It's unlikely if you remember to catch exceptions when calling Lua.

• Can it hang the qbt process with some forever loop and never return?

Yes. Just like a regular function in qBittorrent. The advantage is that you can just stop using a bad plugin (or fix it) instead of waiting for the next update. It is supposed that when using plugins, the responsibility lies with the user himself.

• Is it able to manipulate qbt memory space directly via pointer?

Lua has no low level memory access.

[Chocobo1]

• Can it hang the qbt process with some forever loop and never return?

Yes. Just like a regular function in qBittorrent.

Sounds dangerous? Would it make sense to have a build-time option to enable/disable the lua plugin feature?

The advantage is that you can just stop using a bad plugin

Lets say a newbie user installed a few lua plugins and one of them is faulty. Now qbt is unresponsive immediately after starting it. How can the user find out which one is faulty and remove it?
This is just to assess the current condition and not a goal for this PR.

[glassez]

How can the user find out which one is faulty and remove it?

Well, at least it can disable them all and enable one by one.

[glassez]

• Can it hang the qbt process with some forever loop and never return?

Yes. Just like a regular function in qBittorrent.

Sounds dangerous?

Is it really that dangerous? Well, it will cause the program to freeze, so what? The user always has the option to disable the failed plugin. (If you don't know which one, then you can disable everything.)

[glassez]

Would it make sense to have a build-time option to enable/disable the lua plugin feature?

Then how do you propose to create official builds? With feature enabled, or not?
If enabled, then this option won't make much sense.
If disabled, then you will reduce the audience of the feature to a very small handful of those who use custom builds (I would not like that).

I don't think it makes sense. If users want to use a plugin, they should be aware that we are not responsible for its workability and the overall workability of the application when using it.
If the user is experiencing any problems with the functionality of the application, they should first disable the plug-ins (as well as other additional functions). IMO, these are well-known rules.

[glassez]

It still looks like there is a (indirect) way to control the execution time of Lua code, so I'll try to figure it out. But I still don't consider the current state to be blocking, so I'd rather do this as part of subsequent improvements when this PR is merged.

[glassez]

It still looks like there is a (indirect) way to control the execution time of Lua code, so I'll try to figure it out. But I still don't consider the current state to be blocking, so I'd rather do this as part of subsequent improvements when this PR is merged.

@Chocobo1
I still added the implementation of deadlines for plugins to this PR. The current timeout value is chosen offhand, so as to interrupt the operation of really "looped" plugins, but not to touch the "heavy" ones (it is the user's responsibility whether to use such ones or not). It can be adjusted later when we receive some feedback.

[sledgehammer999]

On the face value of this PR, I don't reject it.
However, I am deeply skeptical of its usefulness going forward. I can't envision a future with many (or at least a few) 3rd party plugins. I can't even imagine what kind of plugins they will be. But we will have the burden of maintaining the code/interface.
Maybe I am too pessimistic. But, I don't reject this PR.

Some thoughts on the technical questions raised (I haven't read any of the code):

1. Maybe it makes sense to run the plugins via a type of event-loop living in a dedicated thread?
2. The plugins are essentially event handlers, right? Consider, implementing a type of ping/pong mechanism between the (management-loop and plugin) to detect stuck plugins and terminate them during runtime. In addition to that, a (long) timeout.
3. Have an explicit cli argument to start qbt with all plugins disabled (aka "safe mode"). In case some plugin gets stuck immediately and prohibits qbt from showing a functioning UI.

Some questions because I've read only the description:

1. These are only lua plugins, right? Not native ones (.dll .so .dylib).
2. These react to events with no ability to mess with the UI or the internal logic of qbt, right? eg make a decision if a torrent should get started/stopped/removed.

[glassez]

1. These are only lua plugins, right? Not native ones (.dll .so .dylib).

Lua only.

2. These react to events with no ability to mess with the UI or the internal logic of qbt, right?

In fact, I intended to provide plugins with the possibility of some functions like start/stop torrents, change some properties like name, comment. Without this, it will hardly be possible to use them for something more or less useful.

[glassez]

I can't even imagine what kind of plugins they will be.

My thoughts on this topic are in the PR description.

[glassez]

But we will have the burden of maintaining the code/interface.

What exactly is bothering you? If it remains unclaimed, it will be worthless. The existing code will not require any care. Well, if it finds its audience, then most likely there will be user requests for its correction, improvement, extension. But what's unexpected about that?

[glassez]

Have an explicit cli argument to start qbt with all plugins disabled (aka "safe mode"). In case some plugin gets stuck immediately and prohibits qbt from showing a functioning UI.

Don't forget that plugin has a deadline, so it will be interrupted anyway.

[glassez]

Maybe it makes sense to run the plugins via a type of event-loop living in a dedicated thread?

As I said above:

they are executed in the main thread, since the qBittorrent core is not multithreaded.

In order for plugins to live in a separate thread, they must either

1. have read-only access to qBittorrent objects (this would require writing appropriate wrappers that store snapshots of the objects), or
2. access to qBittorrent objects through handles that would dispatch requests to/from main thread and care about objects lifetime.

It should be clear that both of these methods are much more troublesome to implement than the current one.
At the same time, the first of them would significantly limit the possible use cases of plug-ins, which I would not like.
I am potentially considering the second one as an option for future improvement.

[sledgehammer999]

In fact, I intended to provide plugins with the possibility of some functions like start/stop torrents, change some properties like name, comment. Without this, it will hardly be possible to use them for something more or less useful.

OK. I was probably overthinking about potential problems and threat level.

But we will have the burden of maintaining the code/interface.

What exactly is bothering you? If it remains unclaimed, it will be worthless. The existing code will not require any care

Forgive me if my following comment is wrong. I haven't read the code so I don't know exactly how the interface is implemented.
A potential "maintainance problem" will be indirect. Any change of the core (new event, removed event, renamed event, altered event) will need to be reflected in the plugin interface system.
But it probably isn't worth it worrying too much about this, since you have volunteered to implement the plugin system. At least for the short term, you take responsibility for maintaining it too.

Have an explicit cli argument to start qbt with all plugins disabled (aka "safe mode"). In case some plugin gets stuck immediately and prohibits qbt from showing a functioning UI.

Don't forget that plugin has a deadline, so it will be interrupted anyway.

Sure. But I think it is good practice to have a killswitch.

Maybe it makes sense to run the plugins via a type of event-loop living in a dedicated thread?

As I said above:

they are executed in the main thread, since the qBittorrent core is not multithreaded.

In order for plugins to live in a separate thread, they must either

1. have read-only access to qBittorrent objects (this would require writing appropriate wrappers that store snapshots of the objects), or

2. access to qBittorrent objects through handles that would dispatch requests to/from main thread and care about objects lifetime.

It should be clear that both of these methods are much more troublesome to implement than the current one. At the same time, the first of them would significantly limit the possible use cases of plug-ins, which I would not like. I am potentially considering the second one as an option for future improvement.

OK. It was a suggestion. I was thinking about the ping/pong mechanism. That's why I made those 2 suggestions.

[sledgehammer999]

Lua library (of the version 5.5 ATTOW)

I can't find a google reference for lua ATTOW. Is this source provided by https://www.lua.org/ ?

Also put appropriate entries in the AUTHORS file.

[xavier2k6]

ATTOW

At The Time Of Writing