Import of germany.osm (~11G) from http://download.geofabrik.de/osm/.

As one can see the following steps are the relevant, most time-consuming
steps:

* Calculation the objects in areas index. "object in areas" calculation must be
  improved by improved the "point is area" algorithm.
* Generation of ways.dat file. This is time consuming because the nodes of a way
  are resolved. Since data is too big to keep everything in memory and
  on-disk algorithm must be used, repeately scanning data files. Instead of this
  the node index file could be used, but we need some performance tests first.

egrep -h "(^\+ Step|^   =>)" <logfile>

+ Step #1 - Preprocess...
   => 128.146 second(s)
+ Step #2 - Generating 'rawnode.idx'...
   => 32.002 second(s)
+ Step #3 - Generating 'rawway.idx'...
   => 8.849 second(s)
+ Step #4 - Generate 'relations.dat'...
   => 67.382 second(s)
+ Step #5 - Generating 'relation.idx'...
   => 1.530 second(s)
+ Step #6 - Generate 'nodes.dat'...
   => 25.975 second(s)
+ Step #7 - Generating 'node.idx'...
   => 1.052 second(s)
+ Step #8 - Generate 'ways.dat'...
   => 752.376 second(s)
+ Step #9 - Generating 'way.idx'...
   => 17.741 second(s)
+ Step #10 - Generate 'area.idx'...
   => 252.107 second(s)
+ Step #11 - Generate 'areanode.idx'...
   => 5.262 second(s)
+ Step #12 - Generate 'region.dat' and 'nameregion.idx'...
   => 774.346 second(s)
+ Step #13 - Generate 'nodeuse.idx'...
   => 229.044 second(s)
+ Step #14 - Generate 'water.idx'...
   => 29.784 second(s)
   => 2325.608 second(s)

The following files have been generated (du --total -h *.idx *.dat):
67M	area.idx
8,0M	areanode.idx
1,5M	nameregion.idx
3,2M	node.idx
88M	nodeuse.idx
98M	rawnode.idx
15M	rawway.idx
204K	relation.idx
20K	water.idx
15M	way.idx
4,0K	bounding.dat
31M	nodes.dat
730M	rawnodes.dat
22M	rawrels.dat
331M	rawways.dat
21M	region.dat
110M	relations.dat
1,2M	wayblack.dat
635M	ways.dat
2,2G	insgesamt

Of these the following are necessary at application runtime:
67M	area.idx
8,0M	areanode.idx
1,5M	nameregion.idx
3,2M	node.idx
88M	nodeuse.idx
204K	relation.idx
20K	water.idx
15M	way.idx
4,0K	bounding.dat
31M	nodes.dat
21M	region.dat
110M	relations.dat
635M	ways.dat

Where the nodeuse.idx currently is only interesting for the internal routing
solution. However some code change is required to do not make the database
class require it. The water.idx is not finished and will likely increase
in future (not not in a way that it changes the overall calculation
drastically).

So the overall size required on disk for a gemrany map without routing
would be:

du --total -h area.idx areanode.idx bounding.dat nameregion.idx node.idx \
nodes.dat region.dat relation.idx relations.dat water.idx way.idx ways.dat

68M	area.idx
8,5M	areanode.idx
4,0K	bounding.dat
1,5M	nameregion.idx
3,2M	node.idx
31M	nodes.dat
21M	region.dat
216K	relation.idx
114M	relations.dat
20K	water.idx
16M	way.idx
646M	ways.dat
907M	insgesamt

Memory usage:
Open the germany map, search for Bonn, search for Dortmund
(top -b | grep <processname>):

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
11244 tim       20   0 1031m  81m  34m S    0  2.7   0:05.38 lt-TravelJinni
11415 tim       20   0  975m  79m  36m S    0  2.6   0:09.19 lt-OSMScout

Most of the memory useage however comes from opening the data files using memory
maped files.

The same, if no memory maped files are used:

 9276 tim       20   0  219m 127m  13m S    0  4.2   0:02.97 lt-TravelJinni

This is the result of the internal statistics dump. Memory usage is rather
high (but was much higher in the past). There is not yet any real measurement
regarding optimal/minimal cache sizes, so possibly by just reducing the cache
sizes some memory can be saved.

Problem is not the in memory cache for nodes, ways/areas and relations but the
various indexes. The currently biggest index is the area node index and should
be next target for optimizations. After that most memory saving can be gained by
improving the NumericIndex that is used for accessing all date files. Possibly
using caching to hold only used index pages in memory is possible here, too.

nodes.dat entries: 330, memory 28000
Index node.idx: 113 entries, memory 4469
ways.dat entries: 1000, memory 121600
Index way.idx: 192 entries, memory 1800
relations.dat entries: 884, memory 79392
Index relation.idx: 268 entries, memory 1868
area.idx entries: 1000, memory 316888
areanode.idx entries: 385, memory 27560
AdminRegion size 77942, locations size 0, memory 2182376
WaterIndex size 18795, memory 18795


