webERP schema initialization seems to have a LOT of repetition
We see a lot of timeouts in installing webERP app.
it was traced to having a very large schema initialization for mysql.
looking at the mysql initialization file, it looks to have vast amounts of repetition.
i just looked for one thing in the file that i see it's repeated. it looks like 2.6MB for that alone by grepping for it.
then, removing duplicates (sort -u does unique sort, removing duplicates), and after that ... the resulting content without duplicates is over 400 times smaller!
$ grep stockmaster sql/mysql/weberp-new.sql > /tmp/before $ sort -u /tmp/before > /tmp/after
the before file is 2640892 bytes long. the after file is 6546 bytes only!
It definitely seems like this file is not properly generated upstream in the app.
#3 Updated by cpg over 7 years ago
looks like there is a table that has all the duplicates taking up the space, called reportlinks.what i did was:
- remove duplicates in that table
- remove the demo sql file
- apply further compression with bzip2
- hosted it in amahi's servers
loading the table now takes much much less time to load.
Also available in: Atom