Project

General

Profile

Bug #1014

webERP schema initialization seems to have a LOT of repetition

Added by cpg over 7 years ago. Updated over 7 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
-
Target version:
-
Start date:
02/24/2013
Due date:
% Done:

0%


Description

We see a lot of timeouts in installing webERP app.

it was traced to having a very large schema initialization for mysql.

looking at the mysql initialization file, it looks to have vast amounts of repetition.

i just looked for one thing in the file that i see it's repeated. it looks like 2.6MB for that alone by grepping for it.

then, removing duplicates (sort -u does unique sort, removing duplicates), and after that ... the resulting content without duplicates is over 400 times smaller!

$ grep stockmaster sql/mysql/weberp-new.sql > /tmp/before
$ sort -u  /tmp/before  > /tmp/after

the before file is 2640892 bytes long. the after file is 6546 bytes only!

It definitely seems like this file is not properly generated upstream in the app.

History

#1 Updated by cpg over 7 years ago

  • Status changed from New to Closed
  • Assignee set to bigfoot65

@bigfoot65 was able to get things optimized big time and the app installs without timeouts now!

#2 Updated by cpg over 7 years ago

  • Status changed from Closed to Assigned
  • Assignee changed from bigfoot65 to cpg

reopening to take a look more in depth.

#3 Updated by cpg over 7 years ago

looks like there is a table that has all the duplicates taking up the space, called reportlinks.

what i did was:
  • remove duplicates in that table
  • remove the demo sql file
  • apply further compression with bzip2
  • hosted it in amahi's servers

loading the table now takes much much less time to load.

#4 Updated by cpg over 7 years ago

  • Status changed from Assigned to Closed

Installation worked, at least on f14. It still takes a few seconds to load the schema, but it worked well.

Also available in: Atom