Subject: INN FAQ Part 3/4: Operating Guide & Tutorial
From: [email-masked] (Tom Limoncelli)
Date: 10 Dec 1994 13:08:24 -0500
Newsgroups: news.software.nntp,news.software.b,news.answers

Posted-By: post_faq 2.10
Archive-name: usenet/software/inn-faq/part3
Last Changed: $Id: FAQ-inn.3,v 1.90 1994/12/10 17:59:38 tal Exp tal $

                  Part 3 of 4
 
INN FAQ Part 1/4: General Information
INN FAQ Part 2/4: Debugging Guide & Tutorial
INN FAQ Part 3/4: Operational and Misc. Questions
                    Day-to-day operational questions
                    INN is running, but i have this small problem...
                    How do i... (Big changes you can make to the system)
                    Bugs in INN 1.5:
                    Bugs in INN 1.4:
                    Bugs in INN 1.3:
                    Bugs in INN 1.2:
INN FAQ Part 4/4: Appendix A: Norman's install guide

------------------------------
 
Subject:  Table Of Contents for Part 3/4
 
=====================================================================
  TABLE OF CONTENTS FOR PART 3/4:  Operational and Misc. Questions
=====================================================================

DAY-TO-DAY OPERATIONAL QUESTIONS:
        How do I create all those directories in the newsspool?
        Safe way to edit the "active" file?
        What's the best way to upgrade to a new version of INN?
        How do I talk to innd from C or Perl?
        After a crash.
Subject:  Listing every article
INN IS RUNNING, BUT I HAVE THIS SMALL PROBLEM...:
        Suddenly my active and history files are owned by root!
        How come my host name comes out twice in the Path line?
        Expire had problems last night, and while I fixed the problem,
        expire reports "Group not matched (removed?) --
                Using default expiration"
        Why doesn't this newsfeeds entry do what I want?
        Why am I forwarding cancel messages for articles in comp.foo
        Debugging someone that is feeding you.
        Feeds suddenly can't connect anymore!
        I'm getting groups sent to me that I don't want.
        When my feeder connects, I get articles but they don't take
                what's waiting for them.
        Directories are being created with wrong permissions.
        Why am I getting alt.sex.pictures even though I have
        More about the "to.*" groups
        What's a decent syslog.conf configuration?
        INN batcher writing "#!rnews 0" separators
        Posting while throttled doesn't work
        "innd: overview! spooling" and the file "overview!" in /usr/spool/news/out.going
        "newgroup" control messages aren't being executed
HOW DO I... (Big changes you can make to the system):
        How do I set up a delayed IHAVE/SENDME over NNTP?
        Can I use gzip with INN?
        What do I do if /var/spool/news is set up over many partitions?
        Archiving expired articles
        INN on one machine, UUCP modem on a different one
BUGS IN 1.4:
        Security Patch
        Looping Select Patch
BUGS IN 1.3:
        7-bit encoded batches are not correctly processed. Why is this?
        NOV (overchan) doesn't work well.
BUGS IN 1.2:
        Why doesn't nntpget work?

=====================================================================
                  DAY-TO-DAY OPERATIONAL QUESTIONS
=====================================================================

------------------------------

Subject:  How do I create all those directories in the newsspool?

Q:  For example, if you receive comp.sys.amiga.applications, do you
have to mkdir /var/spool/news/comp/sys/amiga/applications?

A:  Nope.  innd creates the directory for you the first time you
receive an article for that newsgroup.

------------------------------

Subject:  Safe way to edit the "active" file?

First of all, you could manipulate the active file using the ctlinnd
"newgroup", "rmgroup" and "changegroup" commands.  However, sometimes
you just need to do a lot of editing all at once:

The following sequence is the shortest:

        ctlinnd pause "edit active"
        [do something to the active file]
        ctlinnd reload active "edit active"
        ctlinnd go "edit active"

Simple!  No need to "flush" since the "pause" does that.

> What if I need to delete 3000 lines from my active file?

I would definately edit the active file manually (using
the above procedure).

> What if I need to delete 10 lines from my active file?

For a couple quick changes, I recommend using "ctlinnd".  This is a
little slow because all channels are closed and reopened after each
"rmgroup", "newgroup", and "changegroup".  However, it's easier than
remembering the above sequence.

DO NOT THROTTLE THE SERVER WHEN DOING MULTIPLE rmgroup COMMANDS.  There
is a bug in INN (all versions) that will shred your active file if you
do multiple "rmgroup" messages while the server if throttled.  This is
a common mistake.  People think the "rmgroup"'s will go faster if the
server is throttled.  It will go faster, it will also shred your active
file.

If you have a large number of groups to remove or create, you can use
awk to write a script to do the work for you.

        % cat thelist
        alt.foo.bar
        alt.delete.me
        comp.sys.mac
        % awk <thelist '{ print "ctlinnd rmgroup " $1 }'
        ctlinnd rmgroup alt.foo.bar
        ctlinnd rmgroup alt.delete.me
        ctlinnd rmgroup comp.sys.mac

Now, you can either send the output of that to "| sh -x",
or you can redirect the output to a file, and "source" the
file.

If you want to create a bunch of newsgroups, the awk
command might be like this:

        % awk <thelist \
        '{ print "ctlinnd newgroup " $1 " y user@host" }' | sh -x

------------------------------

Subject:  What's the best way to upgrade to a new version of INN?

First, you should read the README and the Install.ms (yes, read
them both... again).   Things change in new versions.

Second, the README explains how to do an upgrade.  This document
is redundent, but explains the procedure in more detail.

STEP 1:  Copy the values in the old config.data to your new config.data.
You can do this automaticly with this trick:

        % cd config
        % make subst
        % cp config.dist config.data
        % ./subst -f {OLDFILE} config.data
where "{OLDFILE}" names your old config.data file.

STEP 2:  Edit the config.data to see if you want to change any of
the new settings that didn't exist in the old version's config.data
file.

STEP 3: Compile everything:

        % cd $INN
        % make world

STEP 4: When you feel you are ready to install the new files shut the old daemon:

        % ctlinnd shutdown 'upgrade in progress'
        [ kill innwatch by hand if you need to ]

STEP 4: Install the new files:

        % cd $INN
        % make update

STEP 5: Now update all your $INN/site files to be the same as they were
for your old software.  "cd $INN/site ; make diff-installed" will tell
you what's different between the files in /usr/lib/news and $INN/site.
If you only make changes in the $INN/site directory and use "make
install" to copy them into place you'll save your self a lot of
trouble.  Read $INN/site/Makefile for more interesting things that
"make" can do.

STEP 6: When you feel you are ready to install the new $INN/site files:

        # cd $INN/site
        # make install

STEP 7: Re-start the system:

        % sh /usr/lib/news/etc/rc.news

STEP 8: If everything was done right you should be up and running.
Part 2 of the FAQ gives tips on testing your configuration.

------------------------------

Subject:  How do I talk to innd from C or Perl?

Rich Salz says:

If you are writing C, look at doc/inndcomm.3 and include/inndcomm.h;
they include all you need to do any ctlinnd command (in fact, ctlinnd
itself is little more than a call to the library).

Hacking up a Perl subroutine that spoke to innd's Unix-domain control
socket should be fairly straightforward but hasn't yet been written.

------------------------------

Subject:  After a crash.

"What do I do after a system crash?"

INN handles crashes pretty well.  If there are any problems they
get cleaned up by the nightly expire.  About once a month you
might want to run "makehistory -buv" to look for "lost" articles.
Check the man page for "makehistory" for more information.
(The man page for "makehistory" is in the news-recovery man
page until INN 1.4).

------------------------------

Subject:  Listing every article

People often ask for a way to list every file in the newsspool.  There
are a couple ways of doing this.  They work well for INN as well as C
News:

1. Here's the fastest way.  However, it only lists the files that are
actually in the history file and if an article is crossposted it only
gets listed once:

#!/bin/sh
. /usr/lib/news/innshellvars
cd ${SPOOL}
awk '(NF > 2){print $3}' < ${HISTORY} | tr . /

Sorting the output will improve directory cache efficiency.

2. This lists any article file no matter how many links
you have, etc. and even if it is not listed in the history
file:

        cd /var/spool/news
        gfind . -regex '.*/[0-9][0-9]*$' -print

NOTE: GNU find will execute this much faster than the "find" that
comes with most versions of Unix (including SunOS).

3. If you need to do something fancier than what find can do, consider
using perl's find2perl program.  Given a find command line, find2perl
will output the perl code to do the same thing.  You can then modify
the output to do what you want.  For example:

        find2perl . -mtime +30 -name '[0-9][0-9]*$' -exec '/bin/rm {}'

outputs a perl script that deletes any article that is over 30 days old
(except the regular expression is output as wrong...  change it to:

        /^[0-9]+$/ &&

and it should work just fine.

=====================================================================
          INN IS RUNNING, BUT I HAVE THIS SMALL PROBLEM...
=====================================================================

------------------------------

Subject:  Suddenly my active and history files are owned by root!

rc.news runs from root.  After that, everything else should run as
news.  It sounds like you've run news.daily as root by mistake.  Make
sure all your cron jobs run as news and you'll be fine.

If you have an old "cron" system, you might consider replacing yours
with one of the many public domain replacements.  If you can't create
a different "crontab" for each user, the idiom is:

0 * * * * * su news -c '/do/this/as/news'

------------------------------

Subject:  How come my host name comes out twice in the Path line?

The INN server puts its name in the Path line of every article that it
receives.  Obviously, it has to do this.  The default configuration has
inews put the local host in the Path header.  If nobody posts on the
server and you use fully-qualified domain names on your workstations,
then everything works the right way.  (If `hostname` doesn't give an
FQDN on your machine, you can work-around this by setting the "domain"
value in inn.conf; remember that innd never re-reads inn.conf.  You
must "ctlinnd shutdown x" and then re-start the server).  Many people
don't want the client machines to put their name in the Path header.
To do this, set INEWS_PATH to DONT.  Finally, let me say that it is
probably a mistake to have a "pathhost" line on any machine other than
your server if you set INEWS_PATH to DO.  If you doubt this, please
trace the article flow for yourself.  If you are curious about the
effect of INEWS_PATH, read the nroff source -- not the formatted
output -- of doc/inews.1

------------------------------

Subject:  Expire had problems last night, and while I fixed the
                        problem, it still won't run.

When expire starts up it "reserves" the server so that nobody else can
pause or throttle it.  This prevents anyone else from coming in and
modifying the history database.  If expire bails out because of a bad
error (e.g., your expire.ctl has syntax errors) it leaves the server
reserved so that no maintenance will be done until a good expire run has
occurred.  To unblock the server, use the ctlinnd "reserve" command with
an empty string argument.

------------------------------

Subject: expire reports "Group not matched (removed?) --
                Using default expiration"

Expire says:
Group not matched (removed?) alt.techno-shamanism -- Using default expiration
Group not matched (removed?) misc.computers.forsale -- Using default expiration
Group not matched (removed?) de.rec.sf.startrek -- Using default expiration

YOU DID NOTHING WRONG!

That just means that you've removed those newsgroups groups and expire
is slowly removing articles from the spool as they expire.  Eventually
the articles will all have been deleted and so will these messages.

Here's a neat trick to make deleted groups go away at the next
expire.  Using this combination of lines at the *start* of expire.ctl:
        *:A:0:0:0
        *:U:0:7:31
        *:M:0:14:365
will cause groups which are neither moderated nor unmoderated to be
discarded - the only such groups are deleted ones.  Thanks to
[email-masked] (Ian Phillipps) for this tip!

------------------------------

Subject:  Why doesn't this newsfeeds entry do what I want?
                        "foo.com:alt,!alt.sex"

A newsfeeds entry is not a sys file (C News) entry.  Please read
newsfeeds.5.  You might also find the sys2nf program in the frontends
directory useful, as well as the inncheck Perl script that is found in
the samples directory.  The INN Configuration FAQ has cook-book
examples of the steps required to install a NNTP feed, UUCP feed, and
NNTP via nntplink feed.

------------------------------

Subject:  Why am I forwarding cancel messages for articles in comp.foo
                        when I explicitly have !comp.foo in the newsfeeds entry?

Control messages can be explicitly forwarded, so a control message to
comp.foo is forwarded to sites that recieve either comp.foo or control.
Please see the "Control Messages" section of innd.8.  As that
documentation says, you probably want to put "!control" in the
subscription list for most of your newsfeeds.

------------------------------

Subject:  Debugging someone that is feeding you.

David Myers <[email-masked]> suggests that if a neighbor complains
that their feed to you doesn't work: (1) make sure they've read the man
pages, and (2) have them send a copy of their newsfeeds file.

Truly sage advice!

------------------------------

Subject:  Feeds suddenly can't connect anymore!

Q:  How come feeds tell me they can't connect to me any more?

A:  When innd starts up it reads the hosts.nntp file and looks up the
IP addresses for all the entries mentioned there.  The problem is that
this data is dynamic (sometimes people change IP addresses), and innd
never goes back to check.  If your system stays up for days and one of
your feeds changes their IP address (or has a new CNAME), innd will
reject them.  Rich plans to handle this in INN1.5, but for now you
might find it useful to do a "ctlinnd reload hosts.nntp" out of cron
every day or so or when you notice there's a problem.

Here is a sample crontab entry to use: (news should run this)

55 7,12,17,22 * * * /usr/local/newsbin/ctlinnd -s reload hosts.nntp crontab

I hope people vary the time this runs.  If a huge number of INN hosts,
many running NTP so their clocks are within a few ms., all kick off DNS
lookups at exactly the same time, the internet traffic could get
"interesting".  Try setting the minutes value to the time you added
this entry to crontab rather than everyone using "55".  In fact, if
everyone used their birthday plus 1 if they are born on an odd month,
that would spread it out just fine.

------------------------------

Subject:  I'm getting groups sent to me that I don't want.

Tell the system administrator(s) of the machine(s) that feed news to
you to stop sending those groups.  There is no other way to do it.  (In
B or C News, the groups would end up in junk; at least with INN they
are not taking up space.  You should compile with WANT_JUNK set to
DONT).

If the people that feed you use B news or C news, remember that they
don't use a "newsfeeds" file.  They use a file called "sys" which has a
completely different format for specifying newsgroups.

------------------------------

Subject:  When my feeder connects, I get articles but they don't take what's waiting for them.

I hate to say this, but this really shows that you haven't RTFMed very
much.

News is not automaticly bidirectional (it's like SMTP, not UUCP).  If
you want to send things out you will have to make sure that you run
send-nntp or nntpsend from cron.  nntpsend is easier and elsewhere in
this document there are cookbook examples of what to add every time you
set up a new feed.

------------------------------

Subject:  Directories are being created with wrong permissions.

> Question:
>When I received news for /var/spool/news/foo/bar for the first
>time, the directories got created:
>
># ls -lgR foo
>total 1
>d-wx-w-rwx  2 news     news          512 Feb  9 00:03 bar/
>
>What did I do wrong?
>
>##  Mode that directories are created under.
>#### =()<GROUPDIR_MODE          @<GROUPDIR_MODE>@>()=
>GROUPDIR_MODE           2775

  Answer:
You forgot a zero in front of this number, for the C compiler to interpret it
as octal instead of decimal.

------------------------------

Subject:  Why am I getting alt.sex.pictures even though I have
                        "ME:!alt.sex.pictures" in my newsfeeds file?

The active file is the definitive list of what newsgroups you receive.
INN's ME entry is different from C News and B News; please see
newsfeeds.5.  If you do not want to receive alt.sex.pictures, ask the
system(s) that send you news not to send it to you.  (You would have to do
that no matter what news system you are running.)

------------------------------

Subject:  More about the "to.*" groups

(Thanks to [email-masked] (Joseph Malcolm) for supplying
these answers.)

>1) Why did my local INN act on the sendsys posted to to.neighbor?

to.* groups aren't magic to INN.  Your system received the message,
it acted on it.

>2) Why did my neighbor send the cmsg to all of his neighbors?

See 3.

>3) Is is related to having the "control" group in our newsgroups patterns?

Yes.

>   The INN docs say you probably don't want to do this, but they don't say
>   why.

Actually, they do. This is from innd(8):

        Sites may explicitly have the ``control'' newsgroup in their
        subscription  list,  although  it is usually best to exclude
        it.  If a control message is posted to a  group  whose  name
        ends  with  the  four characters ``.ctl'' then the suffix is
        stripped off and what is left is used  as  the  group  name.
        For  example,  a cancel message posted to ``news.admin.misc.ctl''
        will be sent to all sites that subscribe to  ``control''  or
        ``news.admin.misc''.

There is also a pointer to this in newsfeeds(5).

>   But I still need it in my active file, right?

Yes.

------------------------------

Subject:  What's a decent syslog.conf configuration?

The configuration will be different for each site, but here is what
Greg Earle recommends as the lines for the "news.*" related part.
Remember that most syslog's require tabs, not spaces.

Greg's canonical SunOS 4.1.x INN-related syslog.conf entries (which can
be merged into your current configuration):

#
# INN stuff
#
##  Send critical messages to everyone who is logged in and to the console.
news.crit               *
news.crit               /dev/console

##  Log news messages to separate files.
##  Note that each level includes all of the above it.
## =()<news.crit        @<_PATH_MOST_LOGS>@/news.crit>()=
news.crit               /var/log/news/news.crit
## =()<news.err         @<_PATH_MOST_LOGS>@/news.err>()=
news.err                /var/log/news/news.err
## =()<news.notice      @<_PATH_MOST_LOGS>@/news.notice>()=
news.notice             /var/log/news/news.notice

------------------------------

Subject:  INN batcher writing "#!rnews 0" separators

>Outgoing UUCP batches from here are going out with "#!rnews 0" at
>the head of each article.
 
Most common cause:  your newsfeeds entry has "Wnm" not "Wnb".

Other reasons:

batchfiles have something other than a single space between article
filename and size

batchfiles lack size information (all the articles sizes will be read
from the batch file as zero)

------------------------------

Subject:  Posting while throttled doesn't work

>I want to be able to allow my users to be able to post articles when
>innwatch has throttled the system when the spool disk is "full".

Cannot be done in 1.4.

At a minimum, in 1.5 nnrpd will spool the post for the user.

------------------------------

Subject:  "innd: overview! spooling" and the file "overview!" in /usr/spool/news/out.going

> About once a month or so, I get the following warning messages:
> 
> Jan 20 07:20:22 optima innd: overview!:31:proc:9193 cant flush count 14639 Operation would block
> Jan 20 07:20:22 optima innd: overview! spooling 14639 bytes
> 
> And there's a file "overview!" in /usr/spool/news/out.going with stuff in it.
> 
> Should I be doing anything more with this than ignoring it, and maybe
> occasionally deleting it (it just grows)?

This happens because innd is feeding info to overchan faster than
overchan can process it.  The overflow is sent to the file
"overview!".  This file can be deleted, as nnrpd will grab the missing
data out of the articles "manually".  The slow-down won't be noticed.
However, you can "expireover -a" to "fill in the holes".

To prevent this in the future, you need to make overchan run faster.
This is easy to do.  I found that moving the overview files into
/usr/spool/news/over.view made things fast enough that the problem went
away.  Change "_PATH_OVERVIEWDIR" in config.data, recompile, and "make install".
You will need to recompile any newsreaders that read via NFS or
off the local disk.  To move the files to their new home, use a
one-liner like this:
        awk '{ print $1 }' /usr/lib/news/active | tr . / | awk '{ print "mkdir -p /new/location/" $1 ; print "mv " $1 "/.overview /new/location/" $1 "/.overview" }'
to generate a shell scrip that will move your .overview files.

Why does doing all this speed up overchan?  overchan works by opening
the proper .overview file, appending 1 line to it, then closing the
file.  If you have the .overview file in the same directory as 10000
articles then opening the .overview file will take a huge amount of
time.  The open() call literally searches though an average 5000 (half
of 10000) file names to find ".overview".  If you move your .overview
files so that each one is in it's own directory, (say,
/usr/spool/news/over.view/{group}/{name}/.overview) then open() is
searching through 3 files ( ".", "..", and ".overview") to find 1
file.  ( O(N/2) where N=10000 vs. N=3... and you thought those
freshman year CS classes would never be useful!)

There isn't much you can do to make the "append" and "close" steps much
faster, except maybe install a PrestoServe or similar write-cache, and
that won't help very much.

Profiling overchan (with PureSoft's Quantify product) found that the
open() (actually searching the directory to find the .overview file)
was around 80% of the execution time of overchan.  That was reduced to
40% when I moved the .overview files to their own directory.  With the
change, overview's profiling statistics are pretty flat. (which is
good).

If you do not want to move your .overview files, run "expireover -a" to
fix the problem temporarily.  However, it will come back.

DO NOT try feeding the "overview!" file to overchan manually.
        (1) overchan doesn't do any locking and you'll have two overchan's
                running at once.
        (2) overchan only appends to the .overview files.  If you've gotten
                any articles since the "overview!" file was created (you will
                have) then you'll be appending told old entries that are out of
                order.  Your ".overview" files must be in sorted order for the
                other utilities to work right.

------------------------------

Subject:  "newgroup" control messages aren't being executed

> "newgroup" control messages aren't be executed

The usual blame for this is _PATH_EGREP points to a grep that doesn't
understand regular expressions.  For example, GNU grep only understands
regular expressions if it is called "egrep" (i.e. not "gnuegrep" or
"egnugrep").

Make sure you have a link or symlink between egnugrep and egrep.  You
then need to modify config.data so that _PATH_EGREP is
/your/local/path/egrep and NOT /your/local/path/egnuegrep.  Then
recompile and "make install" to have the new binaries and shell
scripts installed.

You also want to check the syntax of your control.ctl file.

=====================================================================
        HOW DO I... (Big changes you can make to the system)
=====================================================================

------------------------------

Subject:  How do I set up a delayed IHAVE/SENDME over NNTP?

Christophe Wolfhugel <[email-masked]> writes:

Having some of your NNTP newsfeeds delayed by a fixed amount of time is
a good way to reduce your bandwith requirements, or a good way to set
up a backup-feed.  By including a Wt flag in your newsfeeds file, INN
will insert timestamp entries in that batchfile, channels, or
exploders.  This timestamp can be used to implement delayed
ihave/sendme processing.  INN's senders (like innxmit) do not use that
data yet.  However, NNTPLINK does support this delayed IHAVE/SENDME
mechanism since release 3.3  (NNTPLINK can be found on
shape.mps.ohio-state.edu:/pub/nntplink/nntplink.tar.gz).

HOW TO DO IT:

The syntax that you would use in your newsfeeds file would be:

        site:*:Tf,Wtmn:

and run this command now and then:

        nntplink -i batchfile -y 300 -b site news.site.fr

The delayed IHAVE/SENDME is expected to allow bandwidth savings in
situations where all sites use nntplink in following topology:

        Your site -- 64k -----------+-----------  Site 1
                                    |               |
                                    |              2mb
                                    |               |
                                    +------------ Site 2

   Site 1 and 2 are in the same metropolitan area, you feed them both.
   With the standard nntplink layout, you generally send all articles
   twice, which is a waste even if you're at 2 Meg/s link and even if
   Site 1 and 2 do nntplinks, you're faster.

   The delayed link would be used between your site and Site 2.  A 2 or
   3 minute delay allows Site 1 to feed Site 2 before you, and in case
   of a Site 1 outage the backup starts nearly immediately.

   Reasonnable delays are still kept as You -> 1 -> 2 should take less
   than one minute (or just 300 ms disk to disk if using nntplink -i ? :)).

Experiences seem to show that a 2 to 3 minutes delay is
a reasonable choice.

Chris

------------------------------

Subject:  Can I use gzip with INN?

(this was written with the help of Michael Brouwer <[email-masked]>)

There are three things that can be effected by using gzip:  Compression
of old logs, compressing batches to send out, and decompressing batches
that come in.

With INN 1.4 all you need to do is change two lines in config.data to
something like this:

COMPRESS                /usr/local/bin/gzip
DOTZ                    .gz

If you rebuild INN with these options set, all logs will be gzipped, and rnews
will use gzip to decompress news.

gzip will automaticly and transparently decompress UNIX Compress, SCO
UNIX Compress (I'm told it's 99% compatible with UNIX Compress), Pack,
and gzip.  Therefore, you can now receive batches compressed with any
of the above listed formats.  Let's say your site is now has "a
universal decompresser".

It has been reported that if you hardlink gzip to be zcat, and make
sure that it is the zcat that INN uses, you can get the "universal
decompresser" without having to use gzip for your logs.  (Though, gzip
for your logs is a big win, so why make trouble for yourself?)

`send-uucp' will still use compress for outgoing batches, so the sites
you feed won't suddenly start getting data they don't understand.

Before you can send gzipped batches, you should make sure that the
sites that you feed have made the above changes so that they have the
"universal decompresser" too.

Edit send-uucp to use gzip instead of compress for certain hosts (see
example of using compress -b12 for the host esac in send-uucp),
outgoing batches will be gzipped.

If you use sendbatch, you will have to edit the file so that COMPRESS
is set to "gzip" and COMPFLAGS is set to "-9vc".

------------------------------

Subject:  What do I do if /var/spool/news is set up over many partitions?

First of all, you can do this by either mounting a filesystem at
/var/spool/news/comp (for example) or by mounting a filesystem anywhere
and making /var/spool/news/comp a symbolic link to the new partition.

Articles will be written as normal, but cross-posts have to be handled
specially now.  Usually INN handles crossposts by writing the article
to the first newssgroup, and then creating hard links to all the other
places where the article should appear.  Hard links do not take up
additional disk space (except making your directories longer).  Hard
links also have the advantage that the file data doesn't get deleted
until the last hard link is gone (and they can be deleted in any
order).  Therefore, you can expire each newsgroup at a different rate,
but the file data won't delete until it is expired from the last
newsgroup.

The problem is that two hard linked files must both be on the same
filesystem (partition).

When INN sees that it can not make a hard link (because an article is
cross-posted across two partitions) it will try to make a symbolic
link.  If your system can not do symbolic links, set HAVE_SYMLINKS to
DONT in your config.data file.  This will make INN write a second
(or third, etc.) copy of the file instead.  (NOTE: INN 1.4 doesn't
make the extra files.  This feature is planned for INN 1.5.)

Anyway, even though INN will automatically create symbolic links, you
have to give expire the "-l" flag so that it will know to modify its
behavior.  Suppose that a message is posted to rec.photo and
alt.cameras and suppose that rec.photo expires more quickly then the
alt group.  If this happens, then you will be left with a dangling
symlink.  The -l flag prevents this from happening by not removing
the file from rec.photos until alt.cameras expire time permits it
from being deleted.

To inform expire that your spool is split across multiple partitions:

In news.daily, change:
        EXPIREFLAGS="-v1"
to read
        EXPIREFLAGS="-v1 -l"

In expirerm, change:
        RMPROC="fastrm -e -u -s ${SPOOL}"
to read
        RMPROC="fastrm -e -s ${SPOOL}"

Now edit innwatch.ctl so that it checks all the spool disks, not just
".".  See the lines with "No space (spool)".

Lastly, edit innstat (the line with the "df") so that all spool disks
are included.  After that, you're done!

If you ever need to run "makehistory" you should pay attention to this
caveat in makehistory(8):
 
        Makehistory does not handle symbolic links.  If the news
        spool area is split across multiple partitions, the follow-
        ing commands should probably be run before the database is
        regenerated:
                cd /usr/spool/news
                find . -type l -print | xargs -t rm

However, if /var/spool/news/comp was a symlink to the partition
that comp is really on, you'll have to reinstall that link.  You
have avoid this by changing the above commands to:
                cd /usr/spool/news
                find . -type l -print | grep '[0-9][0-9]*' | xargs -t rm

------------------------------

Subject:  Archiving expired articles

In <[email-masked]> [email-masked] writes:
>What options do I have in INN for archiving local newsgroups?
>Any help would be appreciated. Any cookbook examples would also help.

See doc/archive.8.  You could also put "never:never:never" in your
expire.ctl file.

Here's a cookbook example of an archive feed:
        # Feed all moderated source postings to an archiver
        source-archive!:!*,*sources*,!*wanted*,!*.d\
                :Tc,Nm,Wn:/usr/local/bin/archive -f -i /usr/spool/news.archive/INDEX

Ulf Kieber <[email-masked]> writes:
The INN 1.4 newsfeeds(5) man page show how to set up a /program/ feed
for archive.  The "archive" program currently does NOT support this
method.  i.e. do not use Tp in "newsfeeds" for an archive feed.

Even if "archive" supported being used as a program feed, you would not
want to use it as such if you intended to use the ``-i'' flag as
archive does not do any file locking on it's index file.  The index
file might get corruped by multiple concurrently running instances of
archive, as may happen with a program feed.

------------------------------

Subject:  INN on one machine, UUCP modem on a different one

Say you have a machine named "newsy" and "modemhead".  Newsy runs INN
but only modemhead has any modems.

Receiving batches on modemhead:  Copy /bin/rnews and
/usr/lib/news/inn.conf to modemhead.  It will work like magic.  When
/bin/rnews runs, it will see that it isn't running on newsy and open an
NNTP connection to newsy and feed the batch (one article at a time) to
newsy... newsy thinks it's just getting a regular NNTP feed.  (which
means modemhead has to be listed in hosts.nntp).

Sending batches via modemhead:  The "sendbatch" program calls $(UUX).
Change ${UUX} to be defined like "rsh modemhead uux" instead of "uux".
You'll have to do a little hacking on sendbatch.  For example, the part
that checks to see if the queue is full might have to be re-written.
Anyway... now the batches will be generated and send via modemhead's
UUCP system.

Pretty neat, eh?

Other advice:

I set UUX to 'rsh uucphost uux' (note no pipe [|]).
Also, we have no 'uuq' command, but even if we did, it would have
        returned bogus info as $SITE is not known to UUCP on newshost.
        Thus I created a stupid 'uuq' that does 'echo 0 0 0 0 0 0 0'
        to satisfy the awk script.  However, we have no way to monitor
        queue length (tho its of little importance to us as we only have
        3 feeds and they are partial)
Finally, the /etc/passwd entry for 'news' on 'uucphost' MUST list /bin/sh;
        /bin/csh results in 'rnews: event not found' and escaping the '!'
        inside sendbatch had no effect.

------------------------------

Subject:  How do I set up inpaths with INN?

inpaths should work just fine with INN as it ships.  However, you can
make it run faster by using the following shell script.  Edit it to
your tastes.  It replaces the long "(cd /var/spool/news ;
/usr/local/bin/gfind . -type f -print | /usr/lib/news/local/inpaths sdl
| /usr/ucb/mail admin,[email-masked] )" which people usually
use.

#!/bin/sh
. /usr/lib/news/innshellvars
cd ${SPOOL}
awk '(NF > 2){print $3}' < ${HISTORY} | tr . / | sort | \
inpaths `innconfval pathhost` | \
${MAILCMD} newsmaster,[email-masked]

If the inpaths people would include this information in the
README, I could delete it from this FAQ.

=====================================================================
                           INFO ABOUT 1.5
=====================================================================

INN 1.5 hasn't been released yet.  However, people constantly mis-type
"INN 1.5" when they mean "INN 1.4".  The bug is in the human posting
the message, not the software.

Release date for 1.5:  Unknown by even the author.

New features in 1.5:

1)  Fix memory leaks in makehistory, thereby speeding it up.

2)  Posting always allowed, even when server is throttled.

3)  Operating systems that require HAVE_UNIX_DOMAIN set to DONT won't
        be so, uh, handicaped.

4)  A global killfile controlled by a function you write in TCL.
        (Start learning TCL now if you want to stay totally hip.)

5)  More interesting ways to control access to nnrpd.

6)  More support for systems that don't have symbolic links.

7)  Free beer.

8)  Just kidding about #6.

None of these features prevent you from having a happy,
healthy and productive life with INN 1.4sec.

DO NOT ASK RICH ABOUT 1.5.  IT WILL BE RELEASED WHEN IT IS DONE, NOT
WHEN SUFFICIENT NUMBERS OF PEOPLE HAVE ASKED WHEN IT WILL BE FINISHED.
Post to news.software.nntp if you have questions about features.

=====================================================================
                             BUGS IN 1.4
=====================================================================

UNOFFICIAL patches for INN 1.4 are available via anonymous FTP at
ftp://ftp.pop.psu.edu/pub/src/news/inn/patches

The file ftp://ftp.pop.psu.edu/pub/src/news/inn/patches/README describes
each of the UNOFFICIAL patches.

The ones that are highly recommended are:

1.4-to-1.4sec     -- Fixes the major security hole in INN.
1.4sec-to-1.4sec2 -- Fixes the remaining known security hole in INN.
select-loop-bug.patch -- Under some circumstances innd can lose track
                of a file descriptor and end up sitting in a select()
                loop.  If your INN suddenly is using up tons of CPU
                time and not getting much done, install this UNOFFICIAL
                patch.  Some OSs are more susecptible to this bug.

There is a replacement for innwatch that is written in Perl.  Get it from
"ftp://ftp.univ-lyon1.fr/pub/unix/news/inn/contrib/innwatch-beta2.pl.gz".
This directory is mirrored on ftp://ftp.pop.psu.edu/pub/src/news/inn/contrib

=====================================================================
                             BUGS IN 1.3
=====================================================================

------------------------------

Subject:  7-bit encoded batches are not correctly processed. Why is this?

Chris Schmidt <[email-masked]> replies:

The decode program that comes with INN up to version 1.3 is broken.
Because of that the last article in a 7bit encoded batch will not
correctly be decoded (the last characters are screwed up).  This is
fixed in INN 1.4.

------------------------------

Subject:  NOV (overchan) doesn't work well.

Correct.  The NOV support in 1.3 didn't have all the bugs worked out.
Don't use NOV under INN 1.3.  Better yet, upgrade to 1.4sec and get all
the benefits!

=====================================================================
                             BUGS IN 1.2
                 (Hey, it's 1994!  Upgrade already!)
=====================================================================

------------------------------

Subject:  Why doesn't nntpget work?

The nntpget in INN 1.2 doesn't work.  Period.  Upgrade to the latest
version of INN.

-- 
        Tom Limoncelli -- [email-masked] (home) -- [email-masked] (work)

                   "I just *LOVE* scanning for life forms!"