<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<META NAME="ROBOTS" CONTENT="INDEX,FOLLOW">
<META name="Revisit-After" content="7 Days">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="GENERATOR" content="Mozilla/4.73 [en] (Win98; U) [Netscape]">
<meta name="description" content="FIDO-HISTORY-PROJECT 3rd Generation - Fidonet Nodelist History Database Project is a project to collect all Fidonet Nodelists and their contents into one SQL database for retrival">
<meta name="Author" content="Public | Ulrich Schroeter">
<META HTTP-EQUIV="imagetoolbar" CONTENT="no">
<meta name="content-language" content="EN">
<meta name="date" content="2013-08-30">
<meta name="KeyWords" content="Fidonet,Nodelist,Pointlist,Database,Fido,Fido-History-Project,Ambrosia">
<title>The Fidonet :: Fidonet Nodelist Database Project</title>
<?php include 'socialshareprivacy.inc'; ?>
<!-- script type="text/javascript" src="js/jfhp.js"></script -->
<!-- link rel="stylesheet" href="fhp.css" type="text/css" -->
<?php
if (file_exists('../ambrosia.css')) {
?>
<link rel="stylesheet" href="/ambrosia.css" type="text/css">
<?php } ?>
<link rel="stylesheet" href="fhp.css" type="text/css">
<!--[if IE 5]>
<link rel="stylesheet" href="iefhp2.css" type="text/css">
<![endif]-->
<?php
if (file_exists('js/jfhp.js')) {
?>
<script type="text/javascript" src="js/jfhp.js"></script>
<?php } ?>
</head>
<body class="fhp2b">
<div class="fhp2">
<a name="Top"></a>
<?php
if (file_exists("head2.inc")) {
include 'head2.inc';
} else {
if (file_exists("../head2.inc")) {
include 'head2.inc';
}
}
// div +2 line 41 and head2.inc +1
?>
<!-- br -->
<div id="header" class="fhp2">
<ul>
<li><a href="/fidonet/fido-history-project.htm">Documentation</a></li>
<li><a href="/fidonet/nodelist.php">Archive View</a></li>
<li><a href="/fidonet/nlarchive2.php">Database Search</a></li>
<li id="current"><a href="/fidonet/fidonet-nodelist-archive.htm">Nodelist Archive DB Structure</a></li>
<li><a href="/fidonet/fidobase.htm">FidoBase Documentation</a></li>
</ul>
</div>
<!-- div style="width: 100%; color: #5F5F5F; margin: 0;" -->
<div class="fhp2">
<!-- background: #104375; -->
<center>
<h2>Welcome to the Fido-History-Project 3rd Generation !</h2></center>
<?php
// echo "Mirros: ";
// use short syntax "//..." for auto port http: or https:
// ambrosia60.dd-dns.de is also ssl enabled :)
if ($_SERVER["HTTP_HOST"]=='ambrosia60.dd-dns.de') { // ($_SERVER['SERVER_ADDR']=="192.168.1.25") {
echo "<center><a href='/fidonet/fido-history-project.htm'><img src='images/mirror_de.png' width='120' border='0' alt='ambrosia60.dd-dns.de' title='ambrosia60.dd-dns.de'></a> ";
echo "<a href='http://fido.ddutch.nl/fidonet/fido-history-project.htm'><img src='images/mirror_nl.png' width='120' border='0' alt='fido.ddutch.nl' title='fido.ddutch.nl'></a></center>";
} else {
echo "<center><a href='http://fido.ddutch.nl/fidonet/fido-history-project.htm'><img src='images/mirror_nl.png' width='120' border='0' alt='fido.ddutch.nl' title='fido.ddutch.nl'></a> ";
echo "<a href='http://ambrosia60.dd-dns.de/fidonet/fido-history-project.htm'><img src='images/mirror_de.png' width='120' border='0' alt='ambrosia60.dd-dns.de' title='ambrosia60.dd-dns.de'></a></center>";
}
?>
<hr>
<!-- table width="97%" border="0" cellspacing="0" cellpadding="0" align="center" summary="Headlines">
<tr>
<td -->
</div>
<!-- div class="fhp" -->
<div class="fhp2b">
<table width="97%" border="0" cellspacing="0" cellpadding="0" align="center" summary="Sub headline">
<tr>
<td colspan="3"> </td>
</tr>
<tr>
<td>
<h1 class="catcheye">
The Fidonet Nodelist Database Project ...
</h1>
<P ALIGN="JUSTIFY"><b>
<font color="#5F5F5F" FACE="tahoma,verdana,sans-serif" >
[<i>19.08.2013</i>] I have a still open task in my work queue
since a long time that didn't make any progress.<br>
This is the migration project to collect all pointlists
into the Nodelist database as well.<br>
Long time I've tried a trial and error concept.<br>
Now I give it a new try by analysing the Nodelist structural
concept, to find answers for questions that did comes out
of exceptional pointlist handling.
</font></b></p>
</td>
<td width="2%">
</td>
</tr>
</table>
<h3>Index</h3>
<a class="naviNAV" href="#chapter1">Chapter 1 - Nodelist database structure</a><br>
<a class="naviNAV" href="#chapter2">Chapter 2 - Pointlist enhancements over Nodelist database structure</a><br>
<a class="naviNAV" href="#chapter3">Chapter 3 - Rules for importing Pointlists</a><br>
<hr>
<h1 id="chapter1">1. Chapter 1 - Nodelist database structure</h1>
<h3 id="sub11">1.1 Structural Concept</h3>
<p>The structure of current Fidonet Nodelist Database collects the
content of all nodelists ever published. The content has been unified.
In database structures viewpoint, this is a real relational database structure concept.
This means, whenever a new node has been listed, the new nodenumber
becomes a new record in the database. This record never changes
despite the fact, the record is listed every week in the current
nodelist. If the speed setting or another nodelist flag changes,
then a new record will be written into the database.<br>
Another table now records all the nodelists ever imported by
Date and the Julian daynumber in the year.<br>
A third table consists of records, that links the nodelist records
with the nodelists ever published.<br>
<br>
<center><img src="images/archivedb1.png" title="Archive DB Structure"><br>
<b><i>Pic. 1 Archive DB structure</i></b></center>
<br>
With this simple structure, there exist a problem in the retrival process:<br>
Records can be found, this is not on discussion. The inner structure of
one nodelist gets detached. So therefor each nodelist record consists
of addtl. reference links.<br>
A nodes uplink maybe a Hub or a Net Host or a Region Host, listed
as Region Independent AKA (RIA) or a Zone Host, listed
as Zone Independent AKA (ZIA).<br>
The Nodelist also includes a substructure, that cannot be read directly
out of an AKA. This is the Region substructure. To get an answer
under which Region a node is listed, you have to look into the nodelist
and find the most close before Region listing. The same applies for
Node records listed under a Hub.<br>
All other reference infos can be retrieved out of the Node AKA:<br>
<center>Zone:Net/Node</center><br>
<br>
So each unified Node record in the Nodelist Database gets enhanced
with a reference:<br>
<ol>
<li>Region under which the Node is listed</li>
<li>Direct Uplink under which a Node is listed (Hub or Net Host ...)</li>
</ol>
<br>
This simplified structure now allows retrival of all Sysops and AKAs
ever listed.</p>
<hr>
<h1 id="chapter2">2. Chapter 2 - Pointlist enhancements over Nodelist database structure</h1>
<h3 id="sub21">2.1 Discussion</h3>
<p>Pointlists follows in principle the Nodelist structure, but only in
the principle.<br>
First, there exist at least 5 different Pointlist formats that are documented under
<a href="/53f8ee03/drv_q/sds/ftsc/FTS-5002.002">FTS-5002 Pointlist Formats</a><br>
Despite the fact, there are all pointlists, the structure is considerable different.<br>
<br>
But there is one in common, what all pointlist formats shares. This is a
reference to a Nodelist record as all Points listed, are listed as downlinks
of a specific Node. So all pointlists have a reference either way to
a Nodelist Node record - the so called <i>Bossnode</i>.</p>
<h3 id="sub22">2.2 Different Pointlist Formats</h3>
<h4 id="sub221">2.2.1 The Boss- and Poss- Format</h4>
<p>So here we're right in the middle of a structural definition.<br>
Each Pointlist point record requires to be referenced to an Uplink
Bossnode that is part of a Pointlist format.<br>
<br>
The easiest ones are the two formats listed under FTS-5002 named the
Boss Format list and the Poss Format list.<br>
Both lists uses the Boss,[AKA] reference modell.<br>
So there is an easy referencing between a Pointlist record and
the related Nodelist record.<br>
The difference in Boss Format and Poss Format list is, that the
Poss Format uses the Nodelist Status <i>Point</i> for all Points
listings where the Boss Format list allows status flags (Hold, Down, Pvt).
So the Boss Format list is the most precise list in also signal a
points status where the Poss Format only includes redundant informations.
We know, we read a Pointlist, this is signaled by the Boss keyword lines.
So all other lines are point listings. No requirement to signal
and flag them as <i>Point</i> in each pointlist line listing.<br>
<br>
For transfering Boss Format or Poss Format pointlists into the
Nodelist Database only the reference changes from a Hub, Host reference
to a Bossnode reference. All other definitions are similar to a
Nodelist record listing.<br>
<br>
At this step, we can rethink, if there is a requirement to use the
<i>Point</i> Nodelist status as a required flag, requiering a transfer
into the database or if we can retrieve this info from another info?<br>
To make the story short:</p>
<div style="margin-left:20px"><p>Each AKA is splitted 4 dimensional: [Zone]:[Net]/[Node].[Point]<br>
where a Node listing is still Point # 0.<br>
All other Pointnumbers are considered to be <i>Points</i>.</p>
</div>
<p> So every time we want to retrieve the info wether a listing is a
Node or a Point listing in the database, we can look into the
Pointnumber field and if its Not Equal 0 then it is a Point listing.
</p>
<div style="margin-left:20px"><p>So therefor we don't require the <i>Point</i> status defined
in the Nodelist status field and we can use the Nodelist status field
for the infos we can retrieve out of Boss Format lists - the real status
of a Point listing.
</p>
</div>
<p>This applies to other Pointlist Formats too, that includes a status field
NE 'Point' for Point listings.
</p>
<h4 id="sub222">2.2.2 The Point- Format</h4>
<p>This format is also known as V7 pointlist format.<br>
<br>
The Point Format list still has merged Nodelist entries together
with Points listings. So the Point Format list can be read as a copy
of a Nodelist with integrated Point records. The reference to a Bossnode
is similar to a listing of a Node under a Hub or Net Host filled
with Point records between each Node lines.<br>
To signal Point lines as Point lines, the Point Format uses
the <i>Point</i> flag as a status field parameter similar to the
Poss Format listed under 2.2.1<br>
Likewise the discussion about the <i>Point</i> flag in a Poss Format
Point list, this status flag is required in a Point Format list, to
differentiate a Node record from a Point record.<br>
<br>
Incorporate Points from a Point Format list is easy if the focus is
an individual point record. It can be easily transfered like the
Boss Format or Poss Format record can be transfered.<br>
But remember - the Point Format list also includes structural a copy of the
Nodelist(!). So a Bossnode record in the Nodelist is also listed with
a Nodelist line in the Point Format list.<br>
Assuming, that a Nodelist record in a Point Format Pointlist
hasn't been updated properly, we potentialy have 2 different listings
for a Node in 2 lists.<br>
The Nodelist line may differ between the Nodelist and the Pointlist listing.<br>
This isn't a big problem, if the Sysop name in a Nodelist record doesn't
much differ from the Sysop name in a node line of the Point Format list.<br>
But considering, that a nodenumber has been re-assigned to another sysop
in the Nodelist, but the Pointlist nether gets updated, now the
Pointlists Node record clashes with the Nodelists Node record. And this
is a problem.<br>
<br>
To solve this problem, a strict analyse in retrieving Pointlists records
is required.</p>
<div style="margin-left:20px"><p>Each Bossnode listing in any Pointlist requires a matching Nodelist
record to keep the database clean from orphaned Bossnode and respective
Points listings.</p>
</div>
<p>This requirement isn't limited to Point Format pointlist records.<br>
<br>
There are some known exceptions, where Point Format pointlists
have shortened Region and/or Host lines. This only requires to
have an exception in analysing input, once readin Region and Host lines
from Point/V7 format pointlists, that such lines ends after field 2
and doesn't follow the FTS Nodelist format lines. For the Bossnode
listings, this doesn't change anything, as Bossnodes are still listed
with complete lines, so they can be compared with the corrosponding
Nodelist record. The only exception is to take care while build up
the AKA from lines that have no comma as seperator after the Region
or Host number but followed by a carriage return sequence.
</p>
<div style="margin-left:20px">
<pre>
Region,31 <- Carriage Return Zone: ?
Host,399 <- Carriage Return Net: 399
,666,Just_a_Node,Simcity,Joe_Grunt,9-012-345-678,9600,VFC Node: 666 AKA: 399/666
Point,1,Point_1,Simcity,Joe_Point,-Unpublished-,300, Node: 666 AKA: 399/666.1
Point,2,Point_2,Simcity,Joes_Brother,9-345-678-901,9600,V34,XX,U,Tbc Node: 666 AKA: 399/666.2
</pre>
</div>
<h4 id="sub223">2.2.3 The Fakenet- Format</h4>
<p>The Fakenet-Format list is a mixture of all previously listed formats.<br>
Its a full featured Nodelist where Bossnode listings have been shifted
to Host listings and Net Pointlistkeeper listings to Region listings.<br>
So that is the reason, why this format is named a <i>Fake</i>net Format list.<br>
The Host- and Region- listings are fakes in relation to Nodelist listings
of Bossnodes.<br>
<br>
The reference to a Nodelist listing of a Bossnode is made in the systemname
field, field #3 of a nodelist line within a Host line.<br>
The definition is, that the systemname has to represent the 2-dimensional
AKA of a Bossnode.<br>
e.g. Bossnode 2:244/1120 is defined 244/1120 in the systemname field.<br>
So a Fakenet Format Bossnode segment can be read in the following way:<br>
<ol>
<li>Read the Host line, extract 2d AKA from systemname field, extract Sysopname, extract City</li>
<li>Search the Nodelist for the 2d AKA found in step 1</li>
<li>Compare Sysopname / City found in step 1 with values found in 2 x<sup>1</sup>)</li>
</ol>
x<sup>1</sup>) If the sysopname matches, all is ok. If the sysopname differs, check sysops surname.
If the sysopname matches, then ok, otherwise check also city. If no match is found in either of these
3 steps, it can be assumed, that the sysop has changed and the referenced pointlist segment
no longer matches to the sysop in the Nodelist.<br>
<br>
Each list is defined a Pointlist for a Net or a Region.<br>
e.g. Points24 is defined as the Fakenet Pointlist for Region 24.<br>
This definition has been made long time ago and the only reference into
Region 24 is the filename. As a Fakenet Pointlist doesn't include
any reference into a specific Zone, a Fakenet Pointlist may reference
to either Zone 1 or Zone 2 or Zone 3 if the list is read outside any
context.<br>
This is a problem in automaticly detecting a Fakenet Format pointlist.<br>
A Fakenet Format Pointlist may have Region and Host lines. So it doesn't
differ much to a Nodelist. The only feature element is a 2 dimensional
AKA definition in the Systemname field in all Host lines of such a list.<br>
<br>
To automaticly identify and referencing a Fakenet Format pointlist
addtl. knowledge requirement is needed to reference a specific
Fakenet Format pointlist to a real Fidonet Network or Region.<br>
<br>
As previously said, the Region 24 (Germany) Fakenet Format pointlist is named
POINTS24, and a Region 23 (Denmark) Fakenet Format pointlist has been named
DK-POINT and a Region 34 (Spain) Fakenet Format pointlist has been named
POINTR34.</p>
<div style="margin-left:20px"><p>So there is one requirement to identify Fakenet Format pointlists.
That is identification by Filename mask.</p>
</div>
<p>Pointlist lines can be retrieved from Fakenet Format lists as they've
been retrieved from Boss Format lists, respective the Nodelist status field.<br>
The reference into a real Fidonet Network or Region is given by an addtl. definition
given in the Filemask definition and reference to a Fakenet Format pointlist.<br>
e.g. Net 244, Region 24.
</p>
<p><b>R23 Fakenet Format Exception</b><br>
In the early days of R23 Fakenet Format pointlists development a 2nd method
to reference Bossnodes to a Nodelist record has been established using
the UBOSS:[2d-AKA] keyword in the Host's line as addtl. Userflag.<br>
<center><b>U</b><i>BOSS:net/node</i></center><br>
This format has only be used in Fakenet Format pointlists of Region 23
in the years from 1992 until 2008. The last R23 Fakenet Format pointlist
distributed is dated: 2008-01-08 (DK-POINT.004)
</p>
<div style="margin-left:20px"><p>Extract the 2d-AKA reference from either
Systemname field (default) or from the UBOSS:[2d-AKA] nodelist flags fields (R34 1992-2008).</p>
</div>
<h4 id="sub224">2.2.4 The Fidouser- Format</h4>
<p>The Fidouser Format list consists of lines with the following format of a record:<br>
<center><Keyword>, <Restofname> [<zone>:]<net>/<node>[.<point>]</center><br>
<br>
Records must be sorted on <Keyword>.<br>
<br>
The zone number may be omitted. Default is the own zone. The
address must start on or after column 46. The record must be
padded with trailing blanks to make all records of equal length.<br>
<br>
The keyword is the last word in the name. Usually this is the
surname, but in case of a compound surname it is the last of them.<br>
<br>
Restofname usually is the first name, but it may be more than one
name and/or prefixes to the surname. Keyword and restofname are
separated by a comma and one space.<br>
<br>
Each line is an individual Point listing with an individual reference to a nodelist record.
Extracting Zone:Net/Node from the line references to the Bossnode.<br>
Addtl. infos like Systemname, City, Nodelistflags aren't available.<br>
So there exist 2 solutions, to complete the Point listing
<ol>
<li>to extract the Nodelist line from the Bossnode and use these infos for the point -or-</li>
<li>to add default values for the missed fields</li>
</ol>
Fidouser Format list is an old style pointlist format. One of the earliest
developments before other formats become common. In these days, Points
did live near by their Bossnodes (same city or around) caused by local
calling areas. A point from Hamburg seldom did call a Bossnode in Munic.</p>
<div style="margin-left:20px"><p>So best practice with some added value is to add the Bossnodes city
to a Fidouser Format pointlist line information that maybe exact correct
or nearly correct in the majority of all cases. Individual cases can be simply ignored.</p>
</div>
<p>This usage adds more valuable information into the transfered pointlist
into the database then all Fidouser Format pointlists get a default
meaningless value:<br>
e.g. City -> Fidouser
</p>
<h3 id="sub23">2.3 The multiple Pointlists case</h3>
<p>The Fidonet Nodelist is the weekly distributed phonebook of all
fidonet members. The compilation consists of the Zone 1 - Zone 6, nowadays
Zone 1 - Zone 4 segments.<br>
Each segment can be distributed seperately, but hasn't in the past, so
the complete Nodelist has been distributed week by week since years.<br>
Pointlist distribution did happen in a different way over years.
The first attempt has been made in the early 2000's to collect regional
pointlist segments together into a combined Zone 2 pointlist segment
that also did include a Zone 6 segment.
From the other Zones until today there is no info available, that ever
pointlist segment distributions did happen.<br>
<br>
At least from the end 80's until the early 2000's Pointlist distribution
evaluated several pointlist distribution formats. Each region had
their own specific distribution format. As also different software
used different pointlist formats, it was common, that several regions
distributed more then one different pointlist formats. It also was common,
that the Pointlist distribution format did change over the years.
At the very end the Fido-History-Project archive collected
many different Pointlist formats, multiple formats within a Regions
and a masterlist - the Zone 2 Pointlist.<br>
<br>
The Nodelist has a uniqueness, that the Pointlists never had.
Nodelist records are unique in one week. Pointlists records aren't.
The Fido-History-Project collected Pointlists from over 10 years
of pointlist distributions, where the masterlist still did not exist.
Also since Zone 2 Pointlist distribution started, not all Pointlists
have been incorporated in the first few years. To have all pointlist
segments incorporated into the Fido-History-Project Nodelist and
Pointlist database archive, several pointlists have to be imported.<br>
<br>
This may result in an overlapping of Pointlist records, e.g. if one
pointlist record has been imported from a regional level Fakenet format
pointlist and the same record in the same week has been imported from
the masterlist.<br>
<br>
We know from Nodelists, that sysops may have several different Nodelist
records, e.g. for administrative AKAs and also on a multiple lines
system. But all these records emerges over weeks. Multiple identical
records from different lists aren't forseen, at least not from
the initial database concept.<br>
<br>
Therefor, the database construct requires an addtl. marker:<br>
From which list does the listed record comes from?</p>
<div style="margin-left:20px">
<pre>
Host 2:244/0 from Nodelist
Hub 2:244/1200 from Nodelist
Node 2:244/1120 from Nodelist
Point 2:244/1120.1 from Region 24 Bossformat list
Point 2:244/1120.1 from Region 24 Fakenet format list
Point 2:244/1120.1 from Zone 2 Bossformat list
</pre>
</div>
<p>From the example above we focus on the AKA's:<br>
From the Nodelist we collect AKA's 2:244/0.0, 2:244/1200.0, 2:244/1120.0<br>
and from the Pointlist(s) we collected 2:244/1120.1, 2:244/1120.1 and 2:244/1120.1<br>
so we collected one record and 2 dupe records from 3 different pointlists.
</p>
<h4 id="sub231">2.3.1 Dupechecking or Not, this is still the question</h4>
<p>Dupe checking for Pointnumbers is one possible option to merge
several listings from several pointlists into one unique Fido-History-Project
Nodelist and Pointlist archive database record with references from several
sources. All what to do is a dupecheck of an existing Point aka for ones
week Nodelist/Pointlist listing.<br>
<br>
In case dupes will be eliminated, how can the references be displayed?<br>
From above example 3 listings of 2:244/1120.1 can be displayed in a table
row by row. But if there is only one row, the question might be, how
these references can be displayed?<br>
<i>Hover</i> info similar to the Nodelist Archive overview, where exceptions
are displayed in red, signals addtl. info if a user hovers the Nodelist
record with the mousepointer.</p>
<div style="margin-left:20px">
<pre>
hovering hovered text display
------------- ----------------------------
Host 2:244/0 from Nodelist
Hub 2:244/1200 from Nodelist
Node 2:244/1120 from Nodelist
Point 2:244/1120.1 from R24PNT, POINTS24, Z2PNT
</pre>
</div>
<p>In the case of unified Point AKA listings, the dupe checking has to
consider, that data integrity requires to be checked, that all dupe records
lists the similar content - Sysopname is identical, City is in one local area.<br>
The biggest problem is, if one of the records doesn't match, the question
will araise, how to proceed? Eliminate the mismatching record? Mark the
mismatching record as mismatched?<br>
Before answering this question, we probably have to ask, how such
mismatches can happen?<br>
Well, a discrepancy between a Nodelist's Bossnode record and a corrosponding
Pointlist Bossnode line in e.g. the Point Format pointlist may happen if
the Nodelist update path is a different update path then the Pointlist
segment update path. But the Bossnode reference into a Nodelist is
still a detected requirement (see section <a href="#sub222">2.2.2 The Point- Format</a>).
A mismatch of one Point AKA from different pointlist sources can be
assumed, that it will never happen, because support of different
Pointlist format list is handled by the Pointlistkeepers. So if they'll
distribute a specific pointlist format, another pointlist format list
will be distributed by the same Pointlistkeeper to support different
Fidonet software pointlist format styles.<br>
<br>
Example: in the early Region 24 pointlist distribution, the Fakenet Format
list distribution was the defined major distribution path. This moved over
to the Bossformat distribution years later. Over the time, both formats
have been distributed in parallel. The content of both files was
identical, because the 2nd format has been converted from #1 distributed
pointlist.<br>
<br>
So before we complicating processing, we can assume, that the major
Pointlists AKA listings from several different pointlist format lists
of one distribution week for a specific area are identical. There may
exist cases, where one Point AKA is listed in one pointlist, but not
in another one. But this problem can be solved, by readin all
different pointlist format lists to complete the archive with
missed records.<br>
<br>
There exist known cases of Point listings under different Bossnodes.
So one may be listed with different Point AKA's. This doesn't differ
to multiple Nodelist listings of a Bossnode who also has a Hub, a Host and/or
RC listing in the Nodelist.<br>
The only difference will be shown in the Nodelist search, where the
search for a sysopname will probably give more results then a result list
that follows a specific AKA search. But this phenomenon we still have
with the Node's Nodelist search.<br>
<br>
In the case of unified Pointlist AKA listings, which lists record will be
give preference?<br>
Example: Point 2:244/1120.1 listing in the Bossformat list shows<br>
,1,AMBROSIA60.1,Frankfurt,Ulrich_Schroeter,+49-69-83837052,300,<br>
and the Fakenet format list shows<br>
,1,244/1120.1,Frankfurt,Ulrich_Schroeter,+49-69-83837052,9600,CM,MO,XA,V32B<br>
Which record we will give the preference?<br>
The Bossformat listing? The Fakenet Format listing, as it was long time
the major distribution version?<br>
Ok, here we have to go deeper into the discovery of current database table
structures.<br>
The table NODELIST lists the different Nodelists by date, year, week<br>
The table NL_NODE lists the different AKAs, the full sysopname and the lastname<br>
The table NL_LINE lists the extracted different Nodelist lines ever listed
with a Nodelist status, systemname, location, phone, baudrate, nodelist flags<br>
The table NL_FRMT still has the different Lists types: Nodelist, Boss, Point, Fakenet
pointlists internal value references<br>
The table NL_ENTRY is an index table that links the Nodelist listed under NODELIST
with the record in the table NL_LINE. Each record in NL_LINE has a reference to
one record in table NL_NODE.<br>
Above example with 2 Point AKA listings will result in two records in table
NL_LINE. Each of these 2 records references to one record in table NL_NODE.<br>
The NL_LINE listing isn't much a problem. But the NL_NODE listing requires
further discovery:<br>
The NL_NODE listing for Nodes consists of detailed reference links. If one
moves from one Hub to another, a new NL_NODE listing will be created
to reference the listing under a different Hub.<br>
At import time, we still have the Bossnodes reference (Net/Node) for a Point,
but what we didn't have is the Bossnodes reference under current Nodelist.
There is still a requirement to check the referenced Bossnode in the
corrosponding Nodelist. So here we have to collect not only, that the
Bossnode listing in current Nodelist is valid, but also what the
reference listing is. This means, under which Region a Bossnode is listed,
under which Hub a Bossnode is listed. If we have collected these infos
from the database, we can also write Point records with these infos
into the database. Problem solved.<br>
If there exist no updates in the Nodelists from one week to the next week,
at least table NL_ENTRY will be filled with the reference links of existing
Nodelist records with the new Nodelist.<br>
<br>
<b>Summary:</b><br>
The accuracy of Pointlist listings depends on the accuracy of Bossnode
references to a corrosponding Nodelist record for the Bossnode.<br>
If this is once checked, the related Pointlist Point AKA records can
be assumed to be correct (minimum of mismatches).<br>
Its more likely, that a complete Bossnode segment is orphaned
then a mismatch of one Pointline will happen.<br>
There may exist different Point sysopnames over the time under one
Bossnode, but they'll evolve over the time. The Point#1 listing in 1999
maybe a different one then in 2009. By search for an AKA this we also see
in Nodelists did happen. By search for a sysopname, the result list
ignores listings with a different sysopname.<br>
So there is a slight preference of unified Pointlist AKA listings. The
only question to answer, how the different references gets displayed.
</p>
<h3 id="sub24">2.4 The corrosponding Nodelist line</h3>
<p>One of the requirements in importing pointlists is the Nodelist
referencing for accuracy of each Bossnodes segment. If a Bossnode is no
longer listed in the Nodelist, a Pointlist segment of that Bossnode
listed in a later pointlist becomes orphaned.<br>
All Nodelists are currently still imported, so that in an import process
for Pointlists, the referenced Nodelist still exist in the database.<br>
But what happens with new distributions?<br>
Often Pointlists will be published the day before or around the
Nodelist will be created. For a region the region check will be made
against the Nodelist regional segment of the upcoming Nodelist. But there
are still cases known of Nodelist's late distributions, where the Nodelist
arives the Pointlistkeepers after weeks. Importing the Pointlist doesn't
make sense, until an accurate Nodelist has been published. So for the
current Pointlist processing and if the check for a nodelist didn't
find a matching Nodelist, has to be delayed until the corrosponding
Nodelist arrives.<br>
There is still a work queue processing of current Nodelists implemented
in the database update process that have to be used for Pointlists
import processing too. So its easy to trigger a Pointlist processing
where still the corrosponding Nodelist isn't available. The simple
solution is to set the Pointlist import to status <i>delayed</i> until
the corrosponding Nodelist import has been processed. Once the Nodelist
arrives and the Nodelist processing has been triggered, the delayed
Pointlist import can be processed thereafter.
</p>
<h1 id="chapter3">3. Chapter 3 - Rules for importing Pointlists</h1>
<table summary="Rules for importing Pointlists" border="1">
<tr>
<td>
1.
</td>
<td>
Each Bossnode listing in any Pointlist requires a matching Nodelist
record to keep the database clean from orphaned Bossnode and respective
Points listings.
</td>
<td>
Check each pointlist revision against a corrosponding
Nodelist revision<br>
Check each bossnode in a pointlist against a corrosponding
Node record in corrosponding Nodelist revision
</td>
</tr>
<tr>
<td>
2.
</td>
<td>
Each AKA is splitted 4 dimensional:<br>
[Zone]:[Net]/[Node].[Point]<br>
where a Node listing is still Point # 0.<br>
All other Pointnumbers are considered to be <i>Points</i>.
</td>
<td>
So therefor we don't require the <i>Point</i> status defined
in the Nodelist status field and we can use the Nodelist status field
for the infos we can retrieve out of Boss Format lists - the real status
of a Point listing.
</td>
</tr>
<tr>
<td>
3.
</td>
<td>
Records in Pointlists maybe shortened
</td>
<td>
Pointlist line checks requires check for comma or line termination
delimiters
</td>
</tr>
<tr>
<td>
4.
</td>
<td>
Fakenet Format pointlists identification follows ...
</td>
<td>
identification by Filename mask
</td>
</tr>
<tr>
<td>
5.
</td>
<td>
Fakenet 2d-AKA references
</td>
<td>
from either Systemname field (default) -or-<br>
from the UBOSS:[2d-AKA] nodelist flags fields
</td>
</tr>
<tr>
<td>
6.
</td>
<td>
Fidouser Format data completion
</td>
<td>
add the Bossnodes city and probably other infos
(Phone, Baud, Nodelistflags)
from the Bossnodes Nodelist info
to a Fidouser Format pointlist line
</td>
</tr>
<tr>
<td>
7.
</td>
<td>
Unified Point AKA listings per Nodelist week
</td>
<td>
Check if Point AKA exist otherwise add new record
</td>
</tr>
<tr>
<td>
8.
</td>
<td>
Pointlists processing requires queuing
</td>
<td>
If a corrosponding Nodelist has been processed
the Pointlist can be imported, otherwise processing
has to be delayed
</td>
</tr>
<tr>
<td>
9.
</td>
<td>
Referencing data source
</td>
<td>
Add a data source type reference
to each Nodelist and Pointlist line
(Nodelist, Points24, Z2PNT)
</td>
</tr>
</table>
<br><br>
<!-- /div -->
<?php
// f1
if (file_exists("footer2.inc")) {
include 'footer2.inc';
} else {
if (file_exists("../footer1.inc")) {
include '../footer1.inc';
}
}
?>
<!-- /div -->
</div>
</div>
</body>
</html>