Changes between Version 18 and Version 19 of dataRepositoryManager

Timestamp:

Oct 17, 2012, 11:34:40 AM (13 years ago)

Author:

Martin Kolman

Comment:

--

dataRepositoryManager

@@ -3 +3 @@
 = modRana data repository document =
 This is a description for a simple program, that should generate a data repository for offline Monav routing data. It should be flexible enough to enable extending it to also handle other data in the future.
+
+== Wokflow ==
+1. download source data
+2. do data processing
+3. package results
+4. move packages to repository
+5. update packages in repository & update manifest
+
+=== Sequential ===
+The initial repository implementation will run all the steps in order.
+
+=== Parallel ===
+As most modern systems have at least two independent CPU cores, the repository should parallelize as many operations as possible.
+
+Parallel workflow:
+1. download source data - single downloading thread (due to Geofabrik download slot limitations)
+2. do data processing - multiple threads possible
+3. package results - multiple packagers
+4. move & publish - single publishing thread (as this is most probably IO bound)
+ 1. move packages to repository
+ 2. update packages in repository & update manifest
+
+
 
 == Requirements ==