Linux memory usage …

I’m not sure if this is right, so this could be conjecture, wild coincidence, or just total bullshit.

To find out how much memory you really need in a Linux machine, here’s my guess at how to do it. The OS will use about 90-95% of your memory whether or not you need it or not.

So, I figured, the memory that’s not used by applications is cache, so here’s my reasoning ….
when running “vmstat 5 5” I get:

[email protected]:~$ vmstat 5 5
procs ———–memory———- —swap– —–io—- –system– —-cpu—-
r b swpd free buff cache si so bi bo in cs us sy id wa
0 0 104 40868 97536 211036 0 0 6 20 53 130 3 5 93 0
0 0 104 40868 97540 211032 0 0 0 14 103 184 0 0 100 0
0 0 104 40852 97540 211032 0 0 0 7 103 200 0 0 100 0
0 0 104 40804 97552 211088 0 0 0 47 112 250 0 1 99 0
0 0 104 40748 97556 211084 0 0 0 33 109 213 0 1 99 0

I have ~200mb for cache, which means my applications probably use the rest – 1gb – 200mb = 800mb. I could probably get away with just running on 800mb opposed to 1gb, right?

[email protected]:~$ cat /proc/meminfo
MemTotal: 1035672 kB
MemFree: 40420 kB
Buffers: 97504 kB
Cached: 211068 kB
SwapCached: 12 kB
Active: 770052 kB
Inactive: 134936 kB
HighTotal: 131008 kB
HighFree: 216 kB
LowTotal: 904664 kB
LowFree: 40204 kB
SwapTotal: 867500 kB
SwapFree: 867396 kB
Dirty: 188 kB
Writeback: 0 kB
Mapped: 636228 kB
Slab: 77832 kB
CommitLimit: 1385336 kB
Committed_AS: 1804444 kB
PageTables: 4280 kB
VmallocTotal: 118776 kB
VmallocUsed: 2984 kB
VmallocChunk: 115412 kB

top output:
Mem: 1035672k total, 1001156k used, 34516k free, 89952k buffers
Swap: 867500k total, 104k used, 867396k free, 203864k cached

One thing I don’t want to do is swap.

If I’m wrong, please let me know! Thanks.

Controlling SPAM with Sendmail

It’s annoying to read. It wastes your time. It wastes you disk space. It can also be a really big problem for mail administrators, especially for those with large networks and many users to look over.

There are many solutions to battle spam, but most administrators are hesitant in the event that there is a good chance of blocking or discarding legitimate and possibly email in the process.

Hopefully this article will give you a better understanding of what is at risk and what you can do about blocking, discarding, or marking spam, increasing your mail performance, save disk space, and save time reading through them.

Well, let’s get started …

First, let’s see that you’re not part of the problem. More specifically, let’s make sure that you’re not helping the spammers by being an open relay. This just means that you don’t allow unauthorized people to relay mail through your mail server. This isn’t likely if you’re using Sendmail 8.9 or above, but if you are using custom rulesets or if you played with the sendmail.cf file yourself, you may want to have this checked out. Let’s also check to see if you’re listed on any blacklists while you’re at it. The site I like to use to test open relays is http://www.ordb.org. There, you just put in your IP address and in a day, you should get the results. You can check if you’re listed on any blacklists by going to http://www.dnsstuff.com in their spam database lookup. If you are listed, you may want to contact the list owner to have yourself removed after a test or with an explanation.
Another thing you can do to protect your users and increase the performance of your system is to add to your mc file:
define(`confPRIVACY_FLAGS’,`authwarnings,novrfy,noexpn,goaway,needmailhelo,restrictqrun’)
This will stop spammers from checking against your machine to see if a user exists, make sure that all mail that comes in starts with the MTA saying “helo,” and allow only root to run the mail queue. Disallowing the features will help increase your performance because your system will no longer respond to a bunch of useless questions.
Let’s start fighting spam!
The risk in blocking spam could be substantial. The last thing you want to do is block an important email that could have meant a lot of business. Of course, the least risky method would be not to block spam at, but what would be the point of this article?
We should see the differences the versions of Sendmail because in as you go up in version, the more you can do against spam and the easier it is.
An example of this is that in Sendmail 8.9 or higher, there’s a:
FEATURE(`accept_unresolvable_domains’)dnl
If this feature is not included in the sendmail configuration, sendmail will not accept mail from domains that do not resolve, meaning that there is no A record or MX record for that domain. This prevents spammers from coming up with fake domain names or at least limits them to only certain fake domain names.
One caveat for this is that as you probably know, some DNS servers do get poisoned or cache false information. For this reason, this can prevent you from getting legitimate email.
In Solaris, this feature is automatically turned on in DOMAIN(solaris-generic). For that reason, in the sendmail.mc file, you may wish to replace
DOMAIN(solaris-generic)dnl
with
DOMAIN(solaris-antispam)dnl
Changing to DOMAIN(solaris-antispam) will remove FEATURE(`accept_unqualified_senders’) from your mc file as well.

SUBJECT BLOCKING
A relatively low risk method of blocking spam is by subject line. You can do this with a simple addition to the mc file before compiling it.
Details for this are here:
http://lantech.geekvenue.net/chucktips/ … index_html
This would be good for blocking viruses that are going around and spam with the same subject line. The subject line here however, would have to be an exact match. Of course, spammers are smarter than this. For this reason, on a lot of spam, you will see that the subject lines have random characters at the end of them. Sendmail allows for regular expression matching as well. To do this, there is an example in the README file in /usr/lib/mail/cf. It’s available in Sendmail 8.10 or above.

ACCESS DATABASE
The next relatively risk-free method of blocking spam is by their envelope from address. In Sendmail 8.8.8 (Solaris 2.6), it is relatively expensive and your list probably shouldn’t be very big and it is also somewhat difficult as well. You can store a list of hosts and a list of email addresses, but they get stored as a list and not a map. Your performance degrades as your list gets longer since the entire file is read and each mail will go through the list of checks as it is processed.
Details for setting this up are here: http://www.Sendmail.org/%7Eca/email/check.html
In Sendmail 8.9 and above (Solaris 7-9), you have what is known as the access database. With this, you can keep a large list of email addresses, domains, subdomains, IP addresses, and even IP networks. Since it is stored in a map (hashed database), regardless of it’s size, Sendmail will look at a map and make one call. (It will take longer to build the map if it’s larger and while the map is being built, you won’t have one in it’s place, but that’s a different discussion.) I would advise keeping a pretty big list. I have a relatively small list of domains and no IP addresses in my list because IP addresses change ownership and there’s always that ever so slight chance that you’re blocking email from a domain that you want email from. I have a pretty big list of email addresses. Some can argue that it’s pointless because the spammers can change their email addresses each time, but I think that it’s still worth having.

To use the access database, in versions 8.9 and higher, simply add to your mc file:
FEATURE(`access_db’,`hash -o /etc/mail/access.db’)dnl
Or
FEATURE(`access_db’,`hash -T -o /etc/mail/access.db’)dnl (in Sendmail 8.12)
Create the file: /etc/mail/access
websculptures.com RELAY
bounced.surecom.com REJECT
[email protected] REJECT
xxxmailgirl.com DISCARD
[email protected] 550 No Such User
You can use the RELAY (if you relay for the domain like being a secondary MX record), REJECT will give an access denied by default, and DISCARD will throw the message into /dev/null. You can also give your own error message and assign it a number. Different numbers are supposed to mean different things – you should follow the error codes listed in RFC 821. I like to use discard because you don’t want spammers to get any more clever in their spamming ways and one way of assuring this is by letting them think that you received the email. After you’re done building the file, you need to build the database. Do this with:
makemap hash /etc/mail/access < /etc/mail/access
You do not need to restart Sendmail for this to take effect.

DNS BLACKLISTS (RBL)
Realtime Blackhole Lists are lists of either mail servers on the Internet are open-relay or known spammers. They are useful in stopping spam because they are lists that you do not have to maintain. They are on the Internet where some are free, some are not. What they do is once a connection is made to your mail server, it will do a DNS lookup on a database to see if it’s listed. If listed, it will return an error message to the client giving him a message. This message is custom, but usually will say something like you are listed on this blackhole list and go to them to get removed.

Detailed instructions on setting this up are here:http://mail-abuse.org/rbl/usage.html

On Sendmail 8.9 or higher, it’s very simple. Add the following lines to your mc file:
FEATURE(`access_db’)dnl
FEATURE(`blacklist_recipients’)dnl
FEATURE(`dnsbl’, `blackholes.mail-abuse.org’)dnl
You can replace “blackholes.mail-abuse.org” with any other services you will be using.
Here’s a pretty extensive list: http://www.declude.com/junkmail/support/ip4r.htm
Be aware that not all of them are known spammers and I’m not sure if the lists are maintained by humans. Using the lists can make you lose legitimate emails. They show no mercy on incompetent email administrators who do not know how to protect their machines from sending out spam.
Another note worth mentioning is that the lists can degrade your performance as well. If you are getting too many mails from different IP addresses for instance or if you are using too many lists and your DNS server is slow, you can have problems. If the list maintainers allow you to download their zone files into your own DNS server, you best do so.

THIRD PARTY PRODUCTS – SPAMASSASSIN & RAZOR w/ PROCMAIL or MIMEDefang
Spamassassin (http://www.spamassassin.org) and Vipul’s Razor (razor.sf.net) are two free and very effective spam-fighting applications. They both require you to compilation however. If you enjoy compiling software and have a spam problem, these tools are well worth the time in setting up. Being effective tools, it would make sense that they are complicated. Spamassassin will go through your mail and look thru it and see if the mail fits a bunch of tests listed here: http://www.spamassassin.org/tests.html. If it fits a particular test, it will assign points to it. So if the email has “sex” in it or if its html is formatted a certain way, it will give or take points. A final value will determine whether or not the mail is spam. You have the power to customize the amount of points it assigns for each test and you also can decide on how many points an email has to have in order to be considered spam. Another nice feature of Spamassassin is that it uses the DNSBL’s. It can see that an email came from a certain IP address and rather than reject the mail outright, it will assign it points and the rest of the email can determine whether or not the email is spam. It can also incorporate Vipul’s Razor. Vipul’s Razor is a “distributed, collaborative, spam detection and filtering network.” Spamassassin can take the Razor’s score into consideration as well.
While Spamassassin and Razor are good applications, you will need to find a way for Sendmail to call these applications and use them. The easiest way I think is with Procmail (http://www.procmail.org). Usage of Procmail however, limits your mail scanning however because it is not done as you are receiving the mail, but after Sendmail has already received the mail and passed onto the Mailer. This means that it would not work for domains that you relay for, only users and aliases on that machine.
Instructions for installing Spamassassin (http://www.spamassassin.org/dist/INSTALL) and Razor (http://razor.sourceforge.net/docs/install.html and http://razor.sourceforge.net/docs/razor-check.html) both include the usage of Procmail. Here are some other cool things you can do with procmail: http://www.uwasa.fi/~ts/info/proctips.html.
For scanning all incoming mail, I think that MIMEDefang (http://www.roaringpenguin.com) is one of the best milter applications available. It checks mail as it is being received and can decide while in transit, whether to relay, deliver, discard, or reject the email. MIMEDefang works well with various antivirus applications and works with Spamassassin. For milter capability however, you should run Sendmail version 8.12 or higher. It was available at the time that 8.11 was out, but we are advised not to use it. Sun’s 8.11.6 version of sendmail does not have milter compiled into it. Sun’s 8.12 Sendmail does, but the operating system does not include the libmilter.a file. For this, you will have to download the source (from http://www.sendmail.org) and compile it from the libmilter directory.
MORE TALK ON SENDMAIL VERSIONS
If you are running Solaris 2.6, you should be running Sendmail 8.8.8. For Solaris 7 and 8, you should run 8.11.6 and for Solaris 9, you should be running 8.12.8. If not, patches are available. The way to determine the version you are running is with the command:
/usr/lib/sendmail –d0.101
If you wish to be running any other version of Sendmail, you can compile it from the source from http://www.sendmail.org. I would like to point out that if you do, you should at least compile in the:
define(`confMAPDEF’, `-DNEWDB’)
This is for hash support in your maps (virtusertable, access_db, genericstable, etc.) You will need the BerkeleyDB (http://www.sleepycat.com) for this.
SOME OTHER SPAM REFERENCES
http://spam.abuse.net/adminhelp/mail.shtml
http://www.digitalanswers.org/check_local

iMS add footer to outbound mails

CONVERSIONS IN-CHAN=*;OUT-CHAN=*;CONVERT yes
2. Create conversions file as specified in imta_tailor:
! Append disclaimer to single part messages if the body part
! is text.
!
in-channel=*; out-channel=*;
in-type=text; in-subtype=*; part-number=1;
parameter-symbol-0=APPARENT_NAME; parameter-copy-0=*;
dparameter-symbol-0=APPARENT_FILENAME; dparameter-copy-0=*;
message-header-file=2; original-header-file=1;
override-header-file=1;
command=”/train/conversion/append_disclaimer.sh footer.txt”
!
! Append disclaimer only to the first part of a multipart message
! if that part is a text message part. (part-number=1.1 is the
! first part of a multipart message).
!
in-channel=*; out-channel=*;
in-type=text; in-subtype=*; part-number=1.1;
parameter-symbol-0=APPARENT_NAME; parameter-copy-0=*;
dparameter-symbol-0=APPARENT_FILENAME; dparameter-copy-0=*;
message-header-file=2; original-header-file=1;
override-header-file=1;
command=”/train/conversion/append_disclaimer.sh footer.txt”
! 3. a shell script for /train/conversion/append_disclaimer.sh
#!/bin/sh
#
# File: append_disclaimer.sh
#
# Usage:
# append_disclaimer.sh [-debug] “name-of-disclaimer-text-file”
#
# References:
# http://docs.sun.com/source/816-6009-10/ … .htm#42323
# http://docs.sun.com/source/816-6009-10/ … .htm#42402
#
#
if [ “$1” = “-debug” ] then
shift
set -x fi DISCLAIMER_FILE=$1 DISCLAIMER_FILE=/train/conversion/${DISCLAIMER_FILE} TAG=”Standard Disclaimer
Appended `date`” cp $INPUT_FILE $OUTPUT_FILE # copy original message part to output destination.
# See if the message was already tagged.
grep “Comments: Standard Disclaimer Appended” $MESSAGE_HEADERS >/dev/null if [ $? -ne 0 ] then
# add a blank line
echo “” >> $OUTPUT_FILE
# append the disclaimer
cat $DISCLAIMER_FILE >> $OUTPUT_FILE # Set a directive so the message will be tagged
echo “OUTPUT_DIAGNOSTIC=\”${TAG}\”” > $OUTPUT_OPTIONS fi
#
# end script.
4. footer.txt – just whatever you want to append to your mails:
*****This footer is appended to the message.***** 5. imsimta refresh — that’s about it!]]>

Installing Openssl/Openssh on Solaris 8

Installing Openssl/Openssh on Solaris 8

Some Compiling NOTES
– If you have problems and decide to start over, run “echo $?” after each command to see if you have errors in your steps
– If you get an error “Cannot find ELF”, it may be because you are using the gnu strip (from binutils). Use the strip that comes with Solaris in /usr/ccs/bin

1) Install compiler (gcc or equivalent – I used Forte Developer 7). You can install gcc with packages SUNWgcmn and SUNWgcc from the Solaris Companion CD or you can get it from sunfreeware.com.

2) path set – cc and make in your path
ie: PATH=/opt/sfw/bin:/usr/ccs/bin:$PATH
The “make” binary is /usr/ccs/bin and if you got gcc from the companion cd, it will be in /opt/sfw/bin (if you got it from sunfreeware.com, it will be in /usr/local/bin)

3) Install patch 112438-01 (reboot the machine after install)

4) Install Openssl (from openssl.org) – latest version as of this writing is 0.96g.
./Config
make
make install

5) Install Openssh (openssh.org) – latest version as of this writing is 3.4p1 – I’m configuring it with pam (so that I can authenticate via ldap) and xauth (so that I can do XForwarding)
./configure –with-pam –with-xauth=/usr/openwin/bin/xauth
make
make install

6) Create a user for ssh
useradd -g nobody -s ‘/usr/bin/false’ sshd

7) If you want XForwarding, in /usr/local/etc/sshd_config, set:
X11Forwarding yes

8) Start the SSH server
/usr/local/sbin/sshd

9) You may want a script to start the ssh server. This is a modified version of the one I took from a source I can’t remember:

#!/sbin/sh
#
# Init file for OpenSSH server daemon
RETVAL=0
prog=”sshd”

# Some functions to make the below more readable
KEYGEN=/usr/local/bin/ssh-keygen
SSHD=/usr/local/sbin/sshd
RSA1_KEY=/usr/local/etc/ssh_host_key
RSA_KEY=/usr/local/etc/ssh_host_rsa_key
DSA_KEY=/usr/local/etc/ssh_host_dsa_key
PID_FILE=/var/run/sshd.pid

do_rsa1_keygen() {
if [ ! -s $RSA1_KEY ]; then
echo -n $”Generating SSH1 RSA host key: ”
if $KEYGEN -q -t rsa1 -f $RSA1_KEY -C ” -N ” >&/dev/null; then
chmod 600 $RSA1_KEY
chmod 644 $RSA1_KEY.pub
success $”RSA1 key generation”
echo
else
failure $”RSA1 key generation”
echo
exit 1
fi
fi
}

do_rsa_keygen() {
if [ ! -s $RSA_KEY ]; then
echo -n $”Generating SSH2 RSA host key: ”
if $KEYGEN -q -t rsa -f $RSA_KEY -C ” -N ” >&/dev/null; then
chmod 600 $RSA_KEY
chmod 644 $RSA_KEY.pub
success $”RSA key generation”
echo
else
failure $”RSA key generation”
echo
exit 1
fi
fi
}

do_dsa_keygen() {
if [ ! -s $DSA_KEY ]; then
echo -n $”Generating SSH2 DSA host key: ”
if $KEYGEN -q -t dsa -f $DSA_KEY -C ” -N ” >&/dev/null; then
chmod 600 $DSA_KEY
chmod 644 $DSA_KEY.pub
success $”DSA key generation”
echo
else
failure $”DSA key generation”
echo
exit 1
fi
fi
}

do_restart_sanity_check()
{
$SSHD -t
RETVAL=$?
if [ ! “$RETVAL” = 0 ]; then
failure $”Configuration file or keys are invalid”
echo
fi
}

start()
{
# Create keys if necessary
do_rsa1_keygen
do_rsa_keygen
do_dsa_keygen

echo -n $”Starting $prog:”
$SSHD
RETVAL=$?
# [ “$RETVAL” = 0 ] && touch /var/lock/subsys/sshd
echo
}

stop()
{
echo -n $”Stopping $prog:”
pkill $SSHD
RETVAL=$?
# [ “$RETVAL” = 0 ] && rm -f /var/lock/subsys/sshd
echo
}

reload()
{
echo -n $”Reloading $prog:”
killproc $SSHD -HUP
RETVAL=$?
echo
}

case “$1” in
start)
start
;;
stop)
stop
;;
restart)
stop
start
;;
reload)
reload
;;
condrestart)
if [ “$RETVAL” = 0 ] ; then
stop
# avoid race
sleep 3
start
fi
# fi
;;
status)
status $SSHD
RETVAL=$?
;;
*)
echo $”Usage: $0 {start|stop|restart|reload|condrestart|status}”
RETVAL=1
esac
exit $RETVAL