How to make use of wget with proxy

GNU wget is a command-line device for downloading recordsdata in a non-interactive means. It is non-interactive as a result of it will possibly execute within the background when the consumer isn’t logged on. This is excellent flexibility with wget as maximum internet browsers require the consumer to be logged in whilst any job is occurring. It can paintings with HTTP, HTTPS, and FTP protocols and likewise helps record obtain by means of HTTP proxies.

Wget command is very similar to curl command and may also be put in on Linux/Unix, Mac OS, Windows. If a obtain job is caught within the halfway, you’ll be able to resume it the place it used to be left off.

What do we duvet?

This information will discover the wget command and discover ways to use it with the Squid proxy server.

Using the wget command to obtain a record

wget is an excessively straight-forward device. When used with none possibility, wget will retrieve the sources from the required url and obtain them to the present operating listing. As an instance, have a look at the under instance:

The above record is downloaded to the listing from the place the wget command used to be used.

Setting proxy for wget

Now allow us to flip to our primary topic nowadays: configure wget with proxy. A proxy server has many benefits, of which safety is the principle fear. In this information, we can be the usage of the Squid proxy server, which is already configured for our community. Here is the configuration of Squid in our case, do exchange it to fit your wishes:

Step 1. We have put in Squid on our Kali Linux device with the IP 192.168.186.161. If you haven’t put in squid but, you’ll be able to do it by way of merely working the command:

Step 2. Make positive that Squid is working with the command:

Step 3. Now alter the configuration of Squid as in keeping with your community. We have set Squid to permit gadgets on our community to make use of squid. For this, you’ll be able to merely open the configuration record:

and upload a line ‘acl localnet src’ along side the IP or community deal with you need to permit get right of entry to for. Here within the image under, you’ll be able to see that we’ve got allowed our community 192.168.186.1/24

Also, to find and set the “http_access” to “allow all” as proven under:

Finally, restart the Squid server with:

Ok, that is sufficient for configuring Squid. We will now transfer directly to configuring our consumer from the place we can be working ‘wget’.

Step 4. In this step, our primary paintings starts for configuring ‘wget’. We have decided on any other device on our community. This is our Ubuntu 20.04 device with IP 192.168.186.150. Now open a terminal right here and open the wget configuration record:

And find the 3 strains:

Step 5. Now uncomment the road you need to make use of on your proxy. In our case we’re the usage of best the https proxy, so we can configure it with the next structure:

In our case, it looks as if:

Step 6. After enhancing the wgetrc record, allow us to transfer on to test if our configuration is operating or now not. Let us once more obtain a record the usage of wget:

This time we will be able to see that the wget makes use of a proxy connection to hook up with the server.

Setting proxy for wget with Squid Authentication

In this segment, we can configure wget to make use of Squid authentication. If we don’t cross the credentials, wget won’t obtain the sources. Let’s do it now.

Step 1. Install the under bundle:

Step 2. Create a passwd record throughout the listing ‘/etc/squid/’

Set the possession of this record Squid consumer proxy:

Step 3. We will now upload a brand new consumer, ‘newuser’ to Squid whose credentials will likely be used for authentication:

The above command will ask to go into a password for the consumer ‘newuser’.

Step 4. Open the Squid configuration record:

Step 5. Now seek for the road containing the string “auth_param basic program” and make it to seem like under:

If you don’t to find any of the strains, then merely upload them.

Step 6. Also, upload the next line:

And under the above line, upload or exchange “http_access allow all” to “http_access deny all” as proven under:

Step 7. Finally, restart the squid carrier:

Now first take a look at run wget with out consumer credentials:

This time, it offers an authentication required error as: “Proxy tunneling failed: Proxy Authentication RequiredUnable to establish SSL connection.”

Now run the wget command with the credential of the consumer we simply added in step 3:

Awesome, this time, the wget command runs like a attraction.

More to be informed…

This completes our demonstration of atmosphere wget with and with out authenticated proxy. Hope you’ve got realized too many new issues on this instructional. One factor to watch out of is that infrequently the configuration of programs might range for various distributions. E.g., Squid could have a moderately other configuration record on Kali Linux and Ubuntu. In such instances, the config record will have to be modified properly.