Monday, June 30, 2008

Restricting downloads with squid!

Many times you may want to deny users from downloading huge files
using the proxy server. This blog will help you do that with squid
in GNU/Linux.

To Readers: All those starting with # are run by root user and ;'s are comments inside the configuration files

Here we are using reply_body_max_size TAG in squid.conf

#vim /etc/squid/squid.conf

acl limitme src 10.0.0.1-10.0.0.50
reply_body_max_size 2097152 allow limitme
reply_body_max_size 0 allow all

By default, there is no restriction on the maximum
file that can be downloaded.

Here limitme acl is allowed only a maximum
of 2 MB download.

2097152 bytes = 2097152/1024 Kbytes => 2048 KB => 2 MB

~mj0vy (sreejith dot anujan at gmail dot com)

Use your bandwidth effectively

This blog tries to help you use your ISP bandwidth wisely! You might be the system administrator of a company who always wanted to limit your fellow workers from downloading the media files, limiting the access of streaming videos which are available in YouTube and wanted to share adequate bandwidth to your SMTP and HTTP Servers. This blog is for those persons.

To Readers: All those starting with # are run by root user and ;'s are comments inside the configuration files

We are going to use the delay_pools TAG in squid.
Before going straight into the configuration, I would like to write some theory.

What exactly are delay pools?

They are simply pools which make a delayed response.
They are essentially bandwidth buckets!
Some of you might have quizzically raised your eyebrows when you read buckets, I know! I too was very much confused about this bucket concept! But I think I can clarify the whole concept for you!

Imagine bandwidth bucket has a normal plastic bucket used to storing water! Instead of water these buckets store bandwidth! Initially it will be full! Initially means when no one is using your bandwidth. When a user requests a page, he will get the respone only if theres enough bandwidth available from the bucket he is using. Bucket actually stores traffic! Bandwidth is expressed in terms of how much data is available in one second, like 1Mb/s (1Mbps)
Traffic is expressed in terms of total data, like 1MB.

Size of bucket determines how much bandwidth is available to a client(s). If a bucket starts out full, a client can take as much traffic as it needs until the bucket becomes empty. Client then recieves bucket allotment at the 'fill rate'. (I will tell about the fill rate later, just remember that word in mind).

There are three types of delay pools.

Class 1 => Single aggregate bucket (Totally shared among the members of the bucket)

Class 2 => To understand it better, assume its applied to Class C networks.
Theres one bucket for each network and 256 individual buckets for each ips of every network. Size of individual bucket cannot exceed the network bucket!

Class 3 => One aggregate bucket, 256 network buckets, 65536 individual buckets. (Class B networks)

Now into configuration,

Firstly we need to define how many delay pools we are doing to declare.

delay_pools 2
This means that we have two delay pools.

delay_class 1 3
This means that the first pool is a class 3 pool (Class B networks)

delay_class 2 1
This means that the second pool is a class 1 pool (Single aggregate bucket)

For each pool we should have a delay_class line.

Now we need to define each pools parameters, like the capacity of each pool and fill rate.

delay_parameters 1 7000/15000 3000/4000 1000/2000
this is delay pool parameters for the pool 1
Pool 1 was a class 3 pool. Class 3 pool has 3 buckets, one aggregate bucket, one for 256 networks and one for 65536 individual ips!

delay_parameters 2 2000/8000
The second pool of type class 1. Class 1 has only one aggregate bucket!

Now whats this 2000/8000?
Each bucket is recognized by its rate/size
Here 8000 means that the maximum capacity of the bucket!
And it refills at the rate of 2000 bytes/second
This means that if the bucket is empty, it takes 4 seconds for the bucket to get full if no clients are accessing it!

If you find a declaration like this,
delay_paramters 2 -1/-1
This means theres no limitation to the bucket!

Now lets take an example.

Our ISP connection is 12Mbps and we want our machines to have a maximum of 4 Mbps at peak time.
The rest we dedicate for SMTP or other production servers. We are going to define only one delay pool of class 1

What is actually 12Mbps?

1Mbps = 1 Megabits per second => 1/8 Megabytes per second (8 bits = 1 byte)
1/8 Megabytes per second => 1/8 * 1024 Kilobytes per second => 128Kilobytes per second => 128KBps
so 1 Mbps => 128 KBps
so 12 Mbps => 128 * 12 = 1536 KBps => 1.5 MBps
To sum up
so 12 Mbps = 12/8 MBps => 1.5 MBps

So with this ISP connection we can download a 6 MB file in 4 seconds!

So here the maximum bandwidth available to machines must be 4 Mbps only! (4Mbps ~ 0.5 MBps ~ 512 KBps)

delay_pools 1
delay_class 1 1
delay_parameters 1 524288/1048576

524288 => 524288/1024 KB => 512 KB => 512/1024 MB => 0.5 MB => 0.5*8 => 4Mb

1048576 => 1048576/1024 KB => 1024 KB => 1 MB

Initially the bucket will be full (1 MB traffic). Now a client makes a request to download a 5 MB file.
It will get the maximum speed(12 Mbps) until it downloads 1 MB, but after that it gets only 0.5 MBps
For 1 MB, it takes 1 second as full bucket is available at first. As the bucket drains, it fills at the rate of 0.5 MBps only.
So 0.5 MBps will only be available after 1 MB has been downloaded!
So the file will get downloaded in 9 seconds. (This is all in theory :P)

There another TAG associated with delay_pools.
delay_initial_bucket_level => this parametes expects a value in percentage(%)
This parameter specifies how much bandwidth is put in each bucket when squid service starts.
By default, the value will be 50%, which means that in the previous example, the client will
download at full speed till the download reaches 0.5 MB

eg:

acl throttled src 10.0.0.1-10.0.0.50
delay_pools 1
delay_class 1 1
delay_parameters 1 524288/1048576
delay_access 1 allow throttled

Note: delay_access is very similar to http_access. It determines which delay pool a request falls into!

Hope this was useful for you!

~mj0vy (sreejith dot anujan at gmail dot com)

Saturday, June 28, 2008

User/Group authentication in squid

User authentication in squid:
------------------------------------------
To Readers: All those starting with # are run by root user and ;'s are comments inside the configuration files

User authentication can be done with the help of the ncsa_auth module and with PAM

With ncsa_aith:
----------------------
This module resides in /usr/lib/squid/ in the name nsca_auth.

edit squid.conf and add the following entries

#vim /etc/squid/squid.conf

acl all src 0.0.0.0/0.0.0.0
;inbuilt acl in squid.conf

acl clients src 10.0.0.1-10.0.0.150
;Here we are declaring an acl in the name
;clients which catches all the ip from 10.0.0.1 to 10.0.0.150

acl users proxy_auth REQUIRED
;another acl in the name users of type proxy_auth
;REQUIRED => accept any valid user.
;proxy_auth works only with the help of external authenticators.
;Here we are going to try, ncsa_auth and PAM.

http_access allow clients users
;Logical AND-ing here, both acls must be satisfied
;for access to be allowed.

http_access deny clients
;If the criteria is not met, deny clients immediately
;instead of traversing till acl 'all'

http_access deny all
;eventually deny all
;Note: by default squid denies all with the acl 'all'

auth_param basic program /var/lib/squid/ncsa_auth /etc/squid/.passwd
;Here we are using the authentication program nsca_auth
;basic => scheme which is used.
;/var/lib/squid/ncsa-auth => external authenticator
;/etc/squid/.passwd => file which contains
; username/password credentials.

save squid.conf and exit

Check the syntax of squid.conf
#squid -k parse /etc/squid/squid.conf
If it shows some problem, rectify it :P

Now we need to make the file which acts as the
username/password database!
Use the htpasswd command from httpd-tools-* rpm.

#htpasswd -c /etc/squid/.passwd mj0vy

enter the password for user mj0vy
when htpasswd is done for the first time we need to give the -c flag so that
it would create the password file. For adding more users to this password file
we should not give the -c flag as it would overwrite the existing users'
accounts.

#htpasswd /etc/squid/.passwd sreejith

restart the squid service and configure clients' browser to listen to squid server
and check whether username and password are prompted while accessing the sites.
Note: Squid is a very heavy service. So instead of going for restarting the squid
with init script, do it like this
#squid -k reconfigure

With PAM Module:
---------------------------
One of the beauty with using PAM is that we can group level deny too!

#vim /etc/squid/squid.conf

auth_param basic program /usr/lib/squid/pam_auth
;Here we dont need to specify the username/password file as its declared
;in the PAM file associated with squid.

#vim /etc/pam.d/squid
Its a single line
auth required pam_listfile.so item=user sense=allow

file=/etc/squid/squid_users onerr=fail

Make sure that /usr/lib/squid/pam_auth is setUID to root!
If not do it
#chmod +s /usr/lib/squid/pam_auth

Now add the UNIX users' name to /etc/squid/squid_users who you want to allow proxy access!

If you want to grant access using UNIX groups, change the PAM file as

Its a single line
auth required pam_listfile.so item=group sense=allow
file=/etc/squid/squid_groups onerr=fail

Make the file /etc/squid/squid_groups and enter the groups who you want to allow access!
Note: Make sure the users who are trying to connect MUST be member of this UNIX group!

Note: As PAM checks /etc/passwd, /etc/group and /etc/shadow, the authentication process will be a bit slow!

chroot vsFTPd

Restricting FTP Users in a chroot-ed Jail:
----------------------------------------------------------
We can specify an explicit list of local users to
chroot to their home directories by specifying,

chroot_list_enable=YES

in /etc/vsftpd/vsftpd.conf
more over we need to specify the list of
users which are to be chroot-ed,
in a file specified by

chroot_list_file.

If chroot_list_file=/etc/vsftpd/chroot_users

then we need to enter the users' names in /etc/vsftpd/chroot_users.
Restart the ftp server and login as those users.
Check whether they CD to other
directories. Theres an additional syntax in vsftpd.conf
which just reverse the default configuration just like
userlist_deny=NO does for access rights.
If, chroot_local_user=YES,
then users who are NOT listed in /etc/vsftpd/chroot_users would only be chrooted.

Monday, June 16, 2008

GNU/Linux as an ADC member

Configuring GNU/Linux as an active directory member:
-----------------------------------------------------------------------------
This time we are doing the reverse of what we did in the last blog. Adding a GNU/Linux machine (RHEL 5.0) to an Win 2k3 Domain Controller.

Configure a M$ Windows 2003 Server as a Domain Controller(DC)
for the domain JOINME.COM. The Active Directory Server name is ads.joinme.com with an ip address of 172.24.10.1
Note: Active Directory howto can be found here
Samba server machine is RHEL 5.0 with SELinux in permissive mode.

To Readers: All those starting with # are run by root user and ;'s are comments inside the configuration files

#vim /etc/samba/smb.conf
;for better performance add this
socket options = TCP_NODELAY SO_RCVBUF=8192 SO_SNDBUF=8192

;name of the domain we want to join
workgroup = JOINME
server string = Member of JOINME.COM

;this samba server is not the local master browser
;for the subnet. local master browser stores all the
;NetBIOS names of all other machines in the subnet.
local master = no

;this server does not act as a Domain Master Browser too.
;DMB stores the NetBIOS names of all
;machines in the network. DMB collates
;browselists from all local master browsers.
domain master = no

;never force an election on startup of the
;machine to become local/domain master browser.
preferred master = no

;do not act as a wins server. Let some other host take up that burden.
wins support = no

;then who is the WINS Server?
wins server = 172.24.10.1

;never try DNS queries when an unregistered NetBIOS name has been found.
dns proxy = no

;the kerberos realm to use
realm = JOINME.COM

;make this machine a member of domain
;in an ADS realm. Kerberos should be installed
;and properly configured.
;join to ADS using the 'net' command
security = ADS

;do username/password validation using the ADS
password server = 172.24.10.1

;the seperator to be used between the
;domain name and the username.
;used in conjuction with pam_winbind.so
winbind separator = +

;winbind should operate without domain component.
;JOINME/mj0vy should be treated as mj0vy.
winbind use default domain = yes

;userid/grpid allocation for mapping
;windows SIDs to unix userids/grpids
idmap uid = 27000-33000
idmap gid = 27000-33000

;donot know why this parameters are used. Will do some
;R&D and come up with the solution soon.
winbind enum users = yes
winbind enum groups = yes

;home directory to be used by the domain users
;when winbind creates the unix password
;database from the windows SIDs.
template homedir = /home/%U
template shell = /bin/bash
log file = /var/log/samba/%m.log

;maximum log file size in KB (5 MB here)
max log size = 5120
printcap name = /etc/printcap
load printers = no

These are the default shares configured already.

[homes]
comment = Home directories for the users.
browseable = no
writeable = yes

[printers]
comment = Connected Printers
path = /var/spool/samba
browseable = no
guest ok = no
writeable = no
printable = yes

Kerberos Configuration:
----------------------------------
The Kerberos system authenticates individual users in a network environment. After authenticatin yourself to kerberos, we can use network facilities (rcp,rsh) without having to present passwords to remote hosts, provided the remote hosts support Kerberos system.When we authenticate with kerberos, we will get an initial kerberos ticket. Kerberos uses this ticket for network utilies as rlogin and rcp. However the tickets expire, privileged tickets, those with 'root' instance expire in a few minutes, others might live for more than a day depending on the policy.
Commands 'kinit' and 'kdestroy' are used to initiate and destroy tickets respectievely.

#vim /etc/krb5.conf
[logging]
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5libs.log
admin_server = FILE:/var/log/kadmind.log

[libdefaults]
default_realm = JOINME.COM
dns_lookup_realm = yes
dns_lookup_kdc = yes
ticket_lifetime = 24h

[kdc]
profile = /var/kerberos/krb5kdc/kdc.conf

[appdefaults]
pam = {
debug = false
ticket_lifetime = 36000
renew_lifetime = 36000
forwardable = true
krb4_convert = false
}

Adding the GNU/Linux machine to Active Directory:
-------------------------------------------------------------------------
Start the samba service.
#/etc/init.d/smb start

Initiate a kerberos ticket
#kinit ADMINISTRATOR@JOINME.COM
enter the administrator password in the DC.
Note: Make sure the GNU/Linux machine and the ADS are
having the same time. Othewise some clock skew error
message may throw up!

Now join the machine to ADS.
#net ads join

Winbind Configuration:
----------------------
#vim /etc/nsswitch.conf
passwd: files winbind
shadow: files
hosts: files winbind

#vim /etc/sysconfig/samba
WINBINDOPTIONS = "-B"

Tweak The Name Service Caching Daemon(NSCD):
----------------------------------------------------------------------
enable-cache passwd no
enable-cache group no
enable-cache hosts no

Now restart the network service
#/etc/init.d/network restart

Configure the pam aware services to use the pam_winbind.so shared object.

#vim /etc/pam.d/login
auth sufficient pam_winbind.so
account sufficient pam_winbind.so
session required pam_mkhomedir.so skel=/etc/skel umask=0022

Now restart the samba and winbind daemon.
#/etc/init.d/smb restart
#/etc/init.d/winbind restart
#chkconfig smb on ; chkconfig winbind on

Testing the whole configuration:
--------------------------------
#getent passwd
#getent group
#wbinfo -u
this will display the domain users
#wbinfo -g
this will display the domain groups
Note: The domain and the users/groups will be
separated by a '+' sign which we specified
with 'winbind separator'

GNU/Linux as PDC for M$ Windows with samba

This blog throws light on configuring Samba as PDC for M$ Windows machines on your network. The Linux distribution used is RHEL 5.0.
Eventhough, this works quite well, samba developers need to work more to make it compatible with M$ Windows ADC. Hope all will enjoy this...

To Readers: All those starting with # are run by root user and ;'s are comments inside the configuration files

#vim /etc/samba/smb.conf
;start global configuration section.
[global]
netbios name = TIKANGA
;this samba machine is given a NETBIOS name.

workgroup = PSEUDO
;this samba server acts a PDC for the domain PSEUDO

encrypt passwords = yes
;encrypted password negotiation

domain master = yes
;the samba server handles browsing elections for the
;domain across multiple subnets


local master = yes
;this makes samba force an election when it starts up.
;this 'oc level' (follows below) parameters makes it win
;the election, as this value is higher
than any other
;M$ server implementations.

;Note: Make sure no other samba machine is set with higher
;'os level' value than this.


preferred master = yes
;this also forces the election at startup

os level = 65

security = user
;samba will prompt for a username and password.
;security = domain | ads is used if another DC
;handles the logons.


domain logons = yes
;this makes samba handle domain logons.

logon path = \\%L\profiles\%u
;this parameter is necessay if we want to support
;roaming profiles for ;win200x/XP/NT clients.

;this actually expands to \\PSEUDO\profiles\username

;we have a sharename profiles coming later
;in the configuration file.


logon script = logon.bat
;name of the MS-DOS batch file which must be executed
;when client logs
on to the domain. The path
;specified is relative to the [netlogon] share

;specified later.


logon drive = L:
;this allows the home directory (if exists) of the user
;to be connected to L: drive under My Computer on client.

time server = yes
;this samba machine advertises itself as a
;time server for the domain.


admin users = mj0vy
;this list of users who have administrative
;privilege in this domain,
such as joining clients
;to the domain and make work the
;machine add script
on the fly.

add machine script = /usr/sbin/useradd -d /dev/null -g 100 -s /bin/false -M %u
;each client is considered as a user and
;are added on the fly as each client tries

;to connect with the administrative user account.

;Start of shares
[netlogon]
path = /etc/samba/netlogon
writable = no
browseable = no
;the [netlogon] share is necessary for samba
;to handle domain logons as

;M$ clients need to contact it during the
;logon process and if this share is not
;present
logon process would fail.
;For security reason, writable and
;browseable permissions are removed.


[profiles]
path = /etc/samba/profiles
browseable = no
writeable = yes
create mask = 0600
directory mask = 0700

;the [profiles] share is used to store
;the roaming profiles of the
users.
;The path points to a directory on the samba
;server where the updated profiles are saved

;on each individual user logons.
;Clients must read and write
;to this share.


[homes]
read only = no
browseable = no
guest ok = no
map archive = yes
;guest logons are disabled and there will
;be no path parameter as it will be
;fetched from /etc/passwd.


[backups]
comment = A Test share to check logon scripts!
path = /backups/samba
valid users = mj0vy sujith sreejith
writeable = yes
create mask = 0765
;this share is actually used to check whether
;the logon script has worked successfully or not!

;In logon.bat file, we will be scpecifying to make
;this share available as K: drive in the
;
My Computer of every client. Sticky Bit file
;permissions are recommended for file security.


Making the users and shares:
------------------------------------------
Add the administrator (mj0vy) who could add machine users on the fly. [admin users = mj0vy]
#useradd -d /dev/null -s /sbin/nologin mj0vy
Lock down his unix accound password for security reasons.
#passwd -l mj0vy

Make him a samba user.
#smbpasswd -a mj0vy
#smbpasswd -e mj0vy (this is not required in higher samba versions)

Make the domain users.
#useradd sujith
#passwd -l sujith
#smbpasswd -a sujith
#smbpasswd -e sujith

#useradd sreejith
#passwd -l sreejith
#smbpasswd -a sreejith
#smbpasswd -e sreejith

Now make the directories for [netlogon], [profiles] and [backups] shares.

#mkdir -m 777 /etc/samba/{netlogon,profiles}
#mkdir -p -m 1777 /backups/samba

Now make logon.bat file inside /etc/samba/netlogon and make it executable by the root user.
Note: Running dos2unix against this file is recommended.

#vim /etc/samba/netlogon/logon.bat
net use K: \\TIKANGA\backups

Now start the samba service
#/etc/init.d/smb start
#chkconfig smb on

Note: running smbtree command gives you a tree like view of all the domains, servers and shares on the servers.

Adding Windows clients:
-----------------------------------
Right click My Computer, Take Computer Name, click change.
In the 'Member of' field enter the samba domain 'PSEUDO'. This will prompt a username and password who could
add this machine to the domain. (mj0vy here). You will be prompted to restart the machine once you joined the Domain.
When the machine comes up, press Ctrl+Alt+Del (hardcore M$ Windows users must be used to this TRAP) and select PSEUDO insead of This Machine.
Try logging in as the samba users.

Hope you enjoyed reading...

~mj0vy