NNtmux indexer install Ubuntu 14.04 guide

Install Ubuntu 14.04 Minimal ISO with user name  nntmux  and only choosing to install SSH server and basic server. I got fed up with bouncing from different web pages to get a clean install, So I put it all in one place mainly for myself and for anyone needing a all in one guide.

Check for any updates just in case and reboot.

sudo apt-get update

sudo apt-get upgrade

sudo apt-get dist-upgrade

sudo reboot

Installing pre-requisite software:

These programs will be used later on to install additional software. They might already be installed on your operating system.

sudo apt-get install software-properties-common python-software-properties git

Adding a repository for the minimum supported PHP version:

NNtmux supports PHP version 5.6 or higher, if your distro does not have 5.6+, you must add a repository.

To add PHP 5.6 or 7 which are the current supported versions of PHP:

sudo add-apt-repository ppa:ondrej/php

sudo apt-get update

 Installing PHP and the required extensions:

Note that some extensions might be missing here,

For PHP 5.6:

     sudo apt-get install php5.6

sudo apt-get install php5.6-cli php5.6-dev php5.6-json php-pear php5.6-gd php5.6-mysql php5.6-pdo php5.6-curl php5.6-common php5.6-mcrypt php5.6-mbstring php5.6-xml

[Mandatory] Apparmor:

    Disabling Apparmor

sudo apt-get purge apparmor

sudo update-rc.d apparmor disable

 You MUST reboot your server after doing this!

sudo reboot

Installing a MySQL server and client:

Percona:

Add the repo key:

sudo apt-key adv --keyserver keys.gnupg.net --recv-keys 1C4CBDCDCD2EFD2A

Create a text file to put in the repo source:

sudo nano /etc/apt/sources.list.d/percona.list

Paste the deb and deb-src lines.

deb http://repo.percona.com/apt trusty main

deb-src http://repo.percona.com/apt trusty main

Exit and save nano (press control+x, then type y and press Enter).

Update your packages and install percona.

sudo apt-get update

sudo apt-get install percona-server-server-5.5 percona-server-client-5.5 libmysqlclient-dev

Configuring MySQL:

Edit my.cnf (the location can vary, type mysqld -v --help | grep -A 1 'Default options' to find all the possible locations):

sudo nano /etc/mysql/my.cnf

Add (or change them if they already exist ; they go under the [mysqld] section) the following:

max_allowed_packet = 16M

group_concat_max_len = 8192

Consider raising the key_buffer_size to 256M to start, later on you can raise this more as your database grows.

 [Mandatory] Create a MySQL user.

Never use the root user for your scripts.

Log in to MySQL with the root user

sudo mysql -u root -p

nntmux is the databasename, you can change this if you want.
Change YourMySQLServerHostName to the hostname of the server.
If your MySQL server is local, use localhost. If remote, try the domain name or IP address. It has been reported 127.0.0.1 does not work for the hostname.
Change YourMySQLUsername for the username you will use to connect to MySQL in NNTmux.
Do not remove the quotes on the name / hostname / password.

GRANT ALL ON nntmux.* TO 'YourMySQLUsername'@'YourMySQLServerHostName' IDENTIFIED BY 'SomePassword';

[Mandatory] Add file permission to your MySQL user.

The ALL permissions doesn't actually grant them all, you MUST add FILE:

GRANT FILE ON *.* TO 'YourMySQLUsername'@'YourMySQLServerHostName';

Exit MySQL:

\q

Installing and configuring a web server:

Nginx:

(Optional) You might want to remove apache2 first:

sudo apt-get remove apache2

Install Nginx:

sudo apt-get install -y nginx

[Optional - Start]
If you wish to install the newest stable version of Nginx, follow these steps.

You can check if your current version is outdated compared to the Nginx downloads page by typing this command:

nginx -v

Add the Nginx PPA:

sudo add-apt-repository ppa:nginx/stable

Update your sources:

sudo apt-get update

Install the latest version of Nginx:

sudo apt-get install -y nginx

[Optional - End]

Install php fpm, which sends the PHP files to Nginx:

sudo apt-get install -y php5.6-fpm

Create a nginx configuration file for your NNTmux website:

sudo nano /etc/nginx/sites-available/NNTmux

Paste the following into the file, change the settings as needed: The server_name must be changed if you want to use a different hostname than localhost. Example: "server_name localhost 192.168.1.29 mydomain.com;" would work on all those 3. The fastcgi_pass can be changed to TCP by uncommenting it, sockets are faster however.

server {
    # Change these settings to match your machine.
    listen 80 default_server;
    server_name localhost;

    # These are the log locations, you should not have to change these.
    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    # This is the root web folder for NNTmux, you shouldn't have to change this.
    root /var/www/NNTmux/www/;
    index index.html index.htm index.php;

    # Everything below this should not be changed unless noted.
    location ~* \.(?:css|eot|gif|gz|ico|inc|jpe?g|js|ogg|oga|ogv|mp4|m4a|mp3|png|svg|ttf|txt|woff|xml)$ {
        expires max;
        add_header Pragma public;
        add_header Cache-Control "public, must-revalidate, proxy-revalidate";
    }

    location / {
        try_files $uri $uri/ @rewrites;
    }

    location ^~ /covers/ {
        # This is where the NNTmux covers folder should be in.
        root /var/www/NNTmux/resources;
    }

    location @rewrites {
        rewrite ^/([^/\.]+)/([^/]+)/([^/]+)/? /index.php?page=$1&id=$2&subpage=$3 last;
        rewrite ^/([^/\.]+)/([^/]+)/?$ /index.php?page=$1&id=$2 last;
        rewrite ^/([^/\.]+)/?$ /index.php?page=$1 last;
    }

    location /admin {
    }

    location /install {
    }

    location ~ \.php$ {
        include /etc/nginx/fastcgi_params;

        # Uncomment the following line and comment the .sock line if you want to use TCP.
        #fastcgi_pass 127.0.0.1:9000;
        fastcgi_pass unix:/var/run/php5.6-fpm.sock;

        # The next two lines should go in your fastcgi_params
        fastcgi_index index.php;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    }
}

Save and exit nano.

If you have changed the fastcgi_pass to tcp (127.0.0.1:9000), you must edit www.conf to listen on it instead of sockets:

sudo nano /etc/php5/fpm/pool.d/www.conf Change listen = /var/run/php5-fpm.sock tolisten = 127.0.0.1:9000 Save and exit nano.

Create a log folder if it does not exist:

sudo mkdir -p /var/log/nginx

sudo chmod 755 /var/log/nginx

Delete the default Nginx site:

sudo unlink /etc/nginx/sites-enabled/default

Make NNTmux the default Nginx site:

sudo ln -s /etc/nginx/sites-available/NNTmux /etc/nginx/sites-enabled/NNTmux

Restart Nginx and php5-fpm:

sudo service nginx restart

sudo service php5.6-fpm restart or sudo /etc/init.d/php5.6-fpm restart




Add your user to the www-data group

Regardless of which web server you use, you should add your user to the www-data group so that you may create/edit files belonging to the group. Replace $USER for your username, if you will not use the current user for nntmux.

sudo usermod -a -G www-data $USER

This requires you to log back in before it takes effect. Do so now or before the Aquiring NNTmux step.

sudo reboot

Configuring PHP:

Open php.ini for the CLI SAPI (for PHP7 /etc/php/7.0/cli/php.ini):

sudo nano /etc/php/5.6/cli/php.ini

Change the following settings:

max_execution_time = 120

The following can be set to -1 if you have a large amount of system RAM (>=8GB):

memory_limit = 1024M

Change your timezone from a list of timezones here. Remove the preceding ;

date.timezone = YourLocalTimezone

Enable error logging (Needed when reporting bugs)

error_reporting = E_ALL

log_errors = On

error_log = php-errors.log

Close and save this file.

Open the Web SAPI php.ini (for PHP7 /etc/php7.0/). If you have installed Apache2: sudo nano /etc/php5/apache2/php.ini If you have installed Nginx: sudo nano /etc/php/5.6/fpm/php.iniChange the settings using the same settings as the CLI SAPI.

Restart Apache2 or PHP-FPM (if you use nginx for example):

sudo service apache2 restart

sudo service php5-fpm restart

Installing extra software:

Unrar:

You can install Unrar from the repositories, but it's quite old (version 4):

sudo apt-get install unrar

You can also install it by downloading the newest version (recommended, as some RAR files require version 5+): Go to http://www.rarlab.com/download.htm, look for the newest unrar version (5.21 when this was written), right click it and copy the link. Replace the link below with the one you copied:

mkdir -p ~/new_unrar && cd ~/new_unrar

wget http://www.rarlab.com/rar/rarlinux-x64-5.2.1.tar.gz

tar -xzf rarlinux*.tar.gz

sudo mv /usr/bin/unrar /usr/bin/unrar4

sudo mv rar/unrar /usr/bin/unrar

sudo chmod 755 /usr/bin/unrar

cd ~/ && rm -rf ~/new_unrar

 7zip:

sudo apt-get install p7zip-full




Mediainfo:

wget http://mediaarea.net/download/binary/libzen0/0.4.33/libzen0_0.4.33-1_amd64.xUbuntu_14.04.deb

wget http://mediaarea.net/download/binary/libmediainfo0/0.7.89/libmediainfo0_0.7.89-1_amd64.xUbuntu_14.04.deb

wget http://mediaarea.net/download/binary/mediainfo/0.7.89/mediainfo_0.7.89-1_amd64.xUbuntu_14.04.deb

sudo dpkg -i  libzen0_0.4.33-1_amd64.xUbuntu_14.04.deb

sudo apt-get install libmms0 

sudo dpkg -i  libmediainfo0_0.7.89-1_amd64.xUbuntu_14.04.deb

sudo dpkg -i  mediainfo_0.7.89-1_amd64.xUbuntu_14.04.deb

Lame:

sudo apt-get install lame

 FFmpeg or avconv:

On newer versions of ubuntu (14.04+) you can install avconv:

sudo apt-get install libav-tools

You can alternatively install ffmpeg:

(manual compilation) https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu

(automated compilation, possibly unmaintained)https://github.com/jonnyboy/installers/blob/master/compile_ffmpeg.sh

yEnc:

You have 4 choices:

simple_php_yenc_decode is a PHP extension written in C++ which offers the best performance of the 3 options. This is quite easy to install so it is worth it. Note: simple_php_yenc_decode is supported only on PHP5.6.

yydecode is an application written in C, it decodes yEnc files, it offers moderate performance. The results vary in terms of performance, this requires disk I/O since the yEnc data has to be written to the disk. Mounting the folder to RAM (tmpfs) improves I/O but is still slower than simple_php_yenc_decode.

Alternatively, NNTmux has a PHP method which decodes yEnc but has very poor performance, this is good if you can't get simple_php_yenc_decode or yydecode installed.

There is a new php-yenc extension, created by niel. You can get it here it supports both PHP5.6 and PHP7.0 (separate deb packages). We highly recommend this extension.

You can change between any of the four at any time in site-edit part of the site if you have issues or want to test performance. If nothing is enetered and you have installed the new php-yenc extension, extension will be used by default

simple_php_yenc_decode:

cd ~/

git clone https://github.com/kevinlekiller/simple_php_yenc_decode

cd simple_php_yenc_decode/

sh ubuntu.sh

cd ~/

rm -rf simple_php_yenc_decode/

yydecode:

cd ~/

mkdir -p yydecode

cd yydecode/

wget http://colocrossing.dl.sourceforge.net/project/yydecode/yydecode/0.2.10/yydecode-0.2.10.tar.gz

tar -xzf yydecode-0.2.10.tar.gz

cd yydecode-0.2.10/

./configure

make

sudo make install

make clean

cd ~/

rm -rf yydecode/

php-yenc:

wget https://github.com/niel/php-yenc/releases/download/v1.2.2/php5.6-yenc_1.2.2-2_amd64.deb

         sudo dpkg -i php5.6-yenc_1.2.2-2_amd64.deb 

Acquiring NNTmux:

Switch your active group to 'www-data'.

sudo newgrp www-data

          exit

Switch to the system's www location.

cd /var/www/

This folder should have been created when apache2 was installed (by php5 dependencies). If, for whatever reason, it did not do:

mkdir -p /var/www

sudo chown www-data:www-data /var/www

sudo chmod 775 /var/www

Install Composer

sudo php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"  

sudo php -r "if (hash_file('SHA384', 'composer-setup.php') === 'e115a8dc7871f15d853148a7fbac7da27d6c0030b848d9b3dc09e2a0388afed865e6a3d6b3c0fad45c48e2b5fc1196ae') { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;"

sudo php composer-setup.php

sudo php -r "unlink('composer-setup.php');"

sudo mv composer.phar /usr/local/bin/composer

sudo composer create-project --no-dev --keep-vcs --prefer-source nntmux/newznab-tmux NNTmux  

Set the permissions:

During the setup (next step of this guide) you should set perms to 777 to make things easier, otherwise you might fail on step 2 of the web install:

sudo chmod -R 755 /var/www/NNTmux/app/libraries

sudo chmod -R 755 /var/www/NNTmux/libraries

sudo chmod -R 777 /var/www/NNTmux/resources

sudo chmod -R 777 /var/www/NNTmux/www

 
     sudo chmod -R 777 /var/www/NNTmux/nntmux/config/ 

   sudo chmod -R 777 /var/lib/php5

Setting up NNTmux:

Open up an internet browser, head to http://IpAddressOfYourServer/install changing IpAddressOfYourServer for the IP of your server.

If this only shows you the web servers default page, you probably need to use a domain name of some kind. Open your local hosts file (on the computer which you are trying to browse from) and add an entry for the target server.

sudo nano /etc/hosts

IpAddressOfYourServer <server_name_used_in_configuration>

Next, head to the edit site section, turn on header compression if your server supports it, put in paths to optional software, like unrar/ffmpeg/etc..

To figure out the path to the optional software, type which followed by the name of the program. Example: which unrar

Signing up to all the API services(Amazon/Trakt/Rotten tomatoes/etc) is recommended, the keys we offer are used by many NNTmux sites and are throttled so you will get almost no results from those.

Indexing:

Manual indexing:

Head to the "view groups" admin section of the site. (http://IpAddressOfYourServer/admin/group-list.php)

Turn on a group or two (alt.binaries.teevee is recommended).

Move to the indexing script location in your terminal:

cd /var/www/NNTmux/misc/update/

Download binary headers from usenet:

php update_binaries.php

When that is complete, create releases and NZB files using those headers:

php update_releases.php 1 true

---

Automatic indexing:

Indexing using the screen sequential scripts:

First install screen, screen can let you run applications in the background while closing your terminal.

sudo apt-get install screen

Move to the screen sequential folder:

cd /var/www/NNTmux/misc/update/nix/screen/sequential/

Run the script using screen:

screen sh simple.sh or screen sh simple-expanded.sh

Now everything will run automatically.

You can detach from the script, allowing you to close your terminal:

Press control and a together, let go and press d : control+a d

If you want to re-attach to screen to see what is going on, type screen -x

Indexing using the Tmux scripts:

Install tmux, tmux is similar to screen but allows to have multiple terminals visible and other features.

sudo apt-get install tmux time

On your website, head to the admin tmux page (http://IpAddressOfYourServer/admin/tmux-edit.php)

Take your time and read through all the options attentively, I will however show the settings I used below.

Set Tmux Scripts Running to yes.

Set Run Sequential to Basic Sequential.

Set Update Binaries to Simple Threaded Update.

Set Update Releases to Update Releases.

Set Postprocess Additional to All.

Set Postprocess Amazon to Yes.

Set Postprocess Non-Amazon to Properly Renamed Releases.

Set Decrypt Hash Based Release Names to All.

Set Update TV and Theater Schedules to yes.

Click on Save Tmux Settings at the bottom of the page.

In your terminal window (CLI), change current working directory to the tmux directory.

cd /var/www/NNTmux/misc/update/nix/tmux/

Start the tmux script.

php start.php

You can now detach from tmux using this keyboard combo(press control and a, let go press d): control+a d

To re-attach to tmux, type tmux attach

---

There are other automated scripts, you can open them in a text editor or ask around / do research to see what they do.

Extra stuff:

IRCScraper:

IRCSraper is a bot that connects to a IRC server to download PreDB information into your NNTmux SQL database.

This helps renaming releases and gives extra information on your releases. Its best to import the pred.db csv file.

sudo wget https://www.dropbox.com/s/qkmgbvmdv9a5w8q/predb_dump_08172015.tar.gz 

sudo gunzip predb_dump_08172015.tar.gz

sudo tar -xvf predb_dump_08172015.tar

sudo php /var/www/NNTmux/misc/testing/PreDB/dump_predb.php remote tmp/predb_dump_08172015.csv

The import takes a long time be patient.

Edit username in each of these settings file and save as settings.php and ircscraper_settings.php, also you only need to change the first username section in each file with the same username

 sudo nano /var/www/NNTmux/misc/IRCScraper/settings_example.php

sudo nano  /var/www/NNTmux/nntmux/config/ircscraper_settings_example.php

---

Comment sharing:

It is possible to share comments between NNTmux and newznab sites.

Conclusion

For questions join IRC, server Synirc channel #tmux