August 24, 2014 at 11:25Web-gui for wget (light)
Previously, there was a description of possible situations where you might need this solution, but let's omit it. The ability to conveniently create remote downloads that are performed by the usual wget (you can safely see their list with ps ), showing progress is not a new idea. And even there are some solutions , but not relevant, as no one has supported them for more than 5 years.
For torrents, everything is simple and trivial - put Transmission or any similar client with a web face. But for links to simple files / pages you need something of your own. Here is a short list of tasks that prompted me to write one:
If you are interested, welcome to cat:
The web interface is built on Javascript + css3 from the client side, and php (selected as the most popular) from the server side. For full work you will need:
As mentioned above - the script written in php acts as the server side. It performs the following tasks:
For his work, he needs ps, wget and kill, respectively. To obtain the value of the download status (by how many percent is completed) the following algorithm is used:
The path for temporary log files is set in the line "define ('tmp_path', '/ tmp');";
The path to the directory where all downloaded files will be saved is set in the line "define ('download_path', BASEPATH. '/ Downloads');".
Convenient paths to ps, wget and kill are available. To do this, just remove the comment at the beginning of the line and indicate your path, for example: "define ('wget', '/ usr / bin / wget');".
It is possible to set restrictions on the download speed from the settings section. The line "define ('wget_download_limit', '1024');" is responsible for this. If you leave it commented out, there will be no restriction.
To determine in the list of tasks launched through the GUI from any other, a specific flag is used, unique to the GUI. It is set in the line "define ('wget_secret_flag', '--max-redirect = 4321');" and it is not necessary to change it unnecessarily. By the way, if you want your other tasks launched from the terminal to be displayed in the web interface - just add this parameter to them. But do not forget that there are also some other parameters that are no less mandatory (depending on the settings).
I tried to write with the most economical attitude to resources and minimal dependence on the system, but I do not have much experience in php, therefore - I will be grateful to the recommendations for optimization.
The script responds to both POST and GET requests. There is no difference between them. Also responds to parameters passed on the command line. For example:
Request:
192.168.1.2/wget/rpc.php?action=add_task&url=http://mirror.yandex.ru/linuxmint/stable/17/linuxmint-17-cinnamon-dvd-64bit.iso
Answer:
{
status: 1,
msg: "Task added",
id: 10910
}
Request:
192.168.1.2/wget/rpc.php?action=get_list
Answer:
{
status: 1,
msg: "Active tasks list",
tasks: [
{
url: " mirror. yandex.ru/linuxmint/stable/17/linuxmint-17-cinnamon-dvd-64bit.iso " ,
progress: 95,
id: 10910
}
]
}
data: ~ # ps -ax | grep wget
10910? S
11378 pts / 0 S + 0:00 grep wget
data: ~ # cat /tmp/wget9720.log.tmp
--2014-08-24 12: 32: 04-- http://mirror.yandex.ru/linuxmint/stable /17/linuxmint-17-cinnamon-dvd-64bit.iso
Resolving mirror.yandex.ru ... 213.180.204.183, 2a02: 6b8 :: 183
Connecting to mirror.yandex.ru | 213.180.204.183 |: 80 ... connected.
HTTP request sent, awaiting response ... 200 OK
Length: 1286799360 (1.2G) [application / octet-stream]
Saving to: `/ DataVolume / shares / Public / Downloads / linuxmint-17-cinnamon-dvd-64bit.iso`
95% [====================================]] 1,225,527,078 1.37M / s eta 35s
data: ~ #
The “new html5 tags” are not used, but the css3 properties are used to design the progress bar of downloads and adaptive. The design is made in a minimalist style without images (the exception is favicon). If there are no tasks, the field for adding a download address is located in the center of the page; if there are tasks, this field moves up the page and tasks are located below.
All requests are asynchronous (without reloading the page). Page Design - Responsive:
The change of state is also displayed in the title of the tab (window):
At the bottom of the page there is a javascript-bookmark (“Download this”), moving it to the browser’s bookmarks bar, you can add new tasks with one click (an active tab will be added on click; if the tab with video file and this “bookmark” will be clicked on the bookmarks bar - a task will be added to download this video file):
All javascript document code is located in the core.js. In the upper part are the main settings:
I don’t see much point in describing the functional points, but I’ll say that the functions are divided into logical groups, the script is not minified, and there are comments.
When you click on F5, a forced update of tasks occurs, the page reloads only by clicking on the update button in the browser.
As changes I will update this post
Link to the project: https://github.com/tarampampam/wget-gui-light
For torrents, everything is simple and trivial - put Transmission or any similar client with a web face. But for links to simple files / pages you need something of your own. Here is a short list of tasks that prompted me to write one:
- I’m watching a movie online using a tablet, but things appear and I would have to save it in order to watch later;
- You need to download the file to the remote server, and you have to start the terminal every time;
- It would be necessary to download an image of fresh linuxmint, but on a home NAS , not a laptop, working for which this idea came;
- During surfing, the task often arises to save the file and share it.
If you are interested, welcome to cat:
System requirements
The web interface is built on Javascript + css3 from the client side, and php (selected as the most popular) from the server side. For full work you will need:
- * nix (at least it was written specifically for this platform, to run under another one, you need the working ports wget , ps and kill );
- php5.x (most likely it will work on php4.x, by the time of publication this had not been tested);
- A browser with Javascript support (and very desirable with css3 support ).
Features and settings of the server side
As mentioned above - the script written in php acts as the server side. It performs the following tasks:
- Getting information about running tasks;
- Cancel running tasks;
- Adding new tasks;
- Returning results in JSON format.
For his work, he needs ps, wget and kill, respectively. To obtain the value of the download status (by how many percent is completed) the following algorithm is used:
- Wget tasks start with the flags "--background" and "--progress = bar: force";
- The output of the download log is made to the file set in the parameter "--output-file = FILE";
- When querying the status of tasks using "ps -ax" we get the file path set to "--output-file = FILE";
- We read the last line of this file, getting the desired value from it regularly.
The path for temporary log files is set in the line "define ('tmp_path', '/ tmp');";
The path to the directory where all downloaded files will be saved is set in the line "define ('download_path', BASEPATH. '/ Downloads');".
Convenient paths to ps, wget and kill are available. To do this, just remove the comment at the beginning of the line and indicate your path, for example: "define ('wget', '/ usr / bin / wget');".
It is possible to set restrictions on the download speed from the settings section. The line "define ('wget_download_limit', '1024');" is responsible for this. If you leave it commented out, there will be no restriction.
To determine in the list of tasks launched through the GUI from any other, a specific flag is used, unique to the GUI. It is set in the line "define ('wget_secret_flag', '--max-redirect = 4321');" and it is not necessary to change it unnecessarily. By the way, if you want your other tasks launched from the terminal to be displayed in the web interface - just add this parameter to them. But do not forget that there are also some other parameters that are no less mandatory (depending on the settings).
I tried to write with the most economical attitude to resources and minimal dependence on the system, but I do not have much experience in php, therefore - I will be grateful to the recommendations for optimization.
The script responds to both POST and GET requests. There is no difference between them. Also responds to parameters passed on the command line. For example:
php ./rpc.php get_listor php ./rpc.php add_task http://goo.gl/5Qi0Xs.An example of a POST request and Json response:
Request:
192.168.1.2/wget/rpc.php?action=add_task&url=http://mirror.yandex.ru/linuxmint/stable/17/linuxmint-17-cinnamon-dvd-64bit.iso
Answer:
{
status: 1,
msg: "Task added",
id: 10910
}
Request:
192.168.1.2/wget/rpc.php?action=get_list
Answer:
{
status: 1,
msg: "Active tasks list",
tasks: [
{
url: " mirror. yandex.ru/linuxmint/stable/17/linuxmint-17-cinnamon-dvd-64bit.iso " ,
progress: 95,
id: 10910
}
]
}
The output of "ps -ax | grep wget "and" cat /tmp/wget{RNDasket.log.tmp ":
data: ~ # ps -ax | grep wget
10910? S
data: ~ # cat /tmp/wget9720.log.tmp
--2014-08-24 12: 32: 04-- http://mirror.yandex.ru/linuxmint/stable /17/linuxmint-17-cinnamon-dvd-64bit.iso
Resolving mirror.yandex.ru ... 213.180.204.183, 2a02: 6b8 :: 183
Connecting to mirror.yandex.ru | 213.180.204.183 |: 80 ... connected.
HTTP request sent, awaiting response ... 200 OK
Length: 1286799360 (1.2G) [application / octet-stream]
Saving to: `/ DataVolume / shares / Public / Downloads / linuxmint-17-cinnamon-dvd-64bit.iso`
95% [====================================]] 1,225,527,078 1.37M / s eta 35s
data: ~ #
Features and settings of the client side
The “new html5 tags” are not used, but the css3 properties are used to design the progress bar of downloads and adaptive. The design is made in a minimalist style without images (the exception is favicon). If there are no tasks, the field for adding a download address is located in the center of the page; if there are tasks, this field moves up the page and tasks are located below.
All requests are asynchronous (without reloading the page). Page Design - Responsive:
The change of state is also displayed in the title of the tab (window):
At the bottom of the page there is a javascript-bookmark (“Download this”), moving it to the browser’s bookmarks bar, you can add new tasks with one click (an active tab will be added on click; if the tab with video file and this “bookmark” will be clicked on the bookmarks bar - a task will be added to download this video file):
All javascript document code is located in the core.js. In the upper part are the main settings:
- “UpdateStatusInterval = 5 * 1000” - interval for updating data on an open tab (be careful with this parameter on weak servers);
- “DebugMode = false” - debugging mode, debugging information is displayed in console.log;
- "Prc = 'rpc.php'" - the path to the server side script.
I don’t see much point in describing the functional points, but I’ll say that the functions are divided into logical groups, the script is not minified, and there are comments.
When you click on F5, a forced update of tasks occurs, the page reloads only by clicking on the update button in the browser.
Installation
- Download or clone the latest version of the repository;
- Unzip to a directory accessible "from the outside";
- Change the path "define ('download_path', BASEPATH. '/ Downloads');" in "rpc.php";
- Open in a browser, check operation. In case of errors - ask a question .
Change history
As changes I will update this post
- It became too lazy to update, here is a link to the current changelog
- 0.0.9 - Added the ability to specify the name of the saved file by adding a line of the form "-> filename.ext" to the URL in the GUI (the full view of the request will be "htttp: //somehost.io/oldfilename.zip -> newfilename.zip "), Added additional input verification
- 0.0.8 - Added directory check for temporary files, added the ability to add several tasks in the GUI, minor improvements (mainly in the visual part)
- 0.0.7 - A test for the existence and permission to write to download_path ( http://goo.gl/I8gYoK ) has been added to the testing function
- 0.0.6 - Call ps aux changed to ps -ewwo pid, args to speed up parsing (issue # 8)
- 0.0.5 - Security updates, an error message was fixed when adding a task, functions return not just the result true | false - but plus a description, support for working with startup parameters from the console (terminal) is added, a server testing function is added (in gui - instead url pass the word "test"), other minor corrections
- 0.0.4 - Added automatic removal of wget log files, minor fixes
- 0.0.3 - Release on github
What to expect:
- Adding authorization and key access;
- Logging (implemented);
- Automatic (or semi-automatic) cleaning of spent log files (partially implemented);
- Extension for the browser (implemented);
- Detailed settings for tasks, such as the path to download and the name of the saved file (partially implemented);
- Removal of not completed tasks;
- ...
MIT License
Copyright © 2014 Paramtamtam
This license allows individuals who receive a copy of this software and related documentation (hereinafter referred to as the “Software”) to use the Software for free without restrictions, including unlimited right to use, copy, modify, add, publish, distribute, sublicense and / or sell copies of the Software, and to permit persons to whom the Software, subject to the following conditions:
Uk The above copyright notice and these terms and conditions must be included in all copies or significant parts of this Software.
THIS SOFTWARE IS PROVIDED "AS IS", WITHOUT ANY WARRANTY OF ANY KIND, EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO THE WARRANTY OF SHEET SHIPPING SHEET. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR CLAIMS FOR DAMAGES, LOSSES, OR OTHER REQUIREMENTS UNDER APPLICABLE CONTRACT, TORT OR OTHERWISE, ARISING FROM HAVING THE RESULT OF OR RELATED TO THE SOFTWARE OR THE USE OF THE SOFTWARE OR OTHER DEALINGS IN THE SOFTWARE.
This license allows individuals who receive a copy of this software and related documentation (hereinafter referred to as the “Software”) to use the Software for free without restrictions, including unlimited right to use, copy, modify, add, publish, distribute, sublicense and / or sell copies of the Software, and to permit persons to whom the Software, subject to the following conditions:
Uk The above copyright notice and these terms and conditions must be included in all copies or significant parts of this Software.
THIS SOFTWARE IS PROVIDED "AS IS", WITHOUT ANY WARRANTY OF ANY KIND, EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO THE WARRANTY OF SHEET SHIPPING SHEET. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR CLAIMS FOR DAMAGES, LOSSES, OR OTHER REQUIREMENTS UNDER APPLICABLE CONTRACT, TORT OR OTHERWISE, ARISING FROM HAVING THE RESULT OF OR RELATED TO THE SOFTWARE OR THE USE OF THE SOFTWARE OR OTHER DEALINGS IN THE SOFTWARE.
Link to the project: https://github.com/tarampampam/wget-gui-light