Difference between revisions of "CDR Performance"

From OpenEMR Project Wiki
 
(49 intermediate revisions by the same user not shown)
Line 1: Line 1:
==Issues==
Several issues have been brought up in the forums, regarding performance of the CDR engine. Here are the forum threads:
Several issues have been brought up in the forums, regarding performance of the CDR engine. Here are the forum threads:
:[http://sourceforge.net/projects/openemr/forums/forum/202504/topic/4676445 Our experience with the new OpenEMR 4.1.0]
*[http://sourceforge.net/projects/openemr/forums/forum/202504/topic/4676445 Our experience with the new OpenEMR 4.1.0]
:[http://sourceforge.net/projects/openemr/forums/forum/202504/topic/4726062 OpenEMR 4.1 is very slow]
*[http://sourceforge.net/projects/openemr/forums/forum/202504/topic/4726062 OpenEMR 4.1 is very slow]
:[http://sourceforge.net/projects/openemr/forums/forum/202506/topic/4733322 Patient Reminders (4.1.0) fails on large db]
*[http://sourceforge.net/projects/openemr/forums/forum/202506/topic/4733322 Patient Reminders (4.1.0) fails on large db]


==Issues==
:Patient Summary page widgets related to CDR are slow
::*Notable, this gets slower as the number of patients in the database increases.
::*Also, the other non-CDR related widgets(with ajax calls) appear to be slowed down by the CDR related widgets(ajax calls).


:CDR engine based reports are slow
Summary:
::*So far, the Administration-Reminders reports has been reported to not work with a patient database size of 4500 users. Suspect this issue will also be encountered in other CDR engine based reports.
*Patient Summary page widgets related to CDR are slow
:*Notable, this gets slower as the number of patients in the database increases(this has been fixed).
:*Also, the other non-CDR related widgets(with ajax calls) appear to be slowed down by the CDR related widgets(ajax calls).


==Plan==
*CDR engine based reports are slow
*Upgrade Jquery, since this should improve the ajax calls from the main patient summary screen, hopefully avoiding the CDR-unrelated ajax calls from being effected by the CDR-related ajax calls. Note this has already been done by yehster in the 'master' branch: http://github.com/openemr/openemr/commit/a23f3e34e5410f4cdd61f3304b9e0900a8c3e699
:*So far, the Administration-Reminders reports has been reported to not work with a patient database size of 4500 users. Suspect this issue will also be encountered in other CDR engine based reports. (this has been fixed, albeit the reports take a very long time)
:*Awaiting feedback if this helps performace of the larger databases.


==Plan/Fixes==
===Jquery Optimization===
*Upgrade Jquery, since this should improve the ajax calls from the main patient summary screen, hopefully avoiding the CDR-unrelated ajax calls from being effected by the CDR-related ajax calls. Note this has already been done by yehster in the 'master' branch(and is in version 4.1.1): http://github.com/openemr/openemr/commit/a23f3e34e5410f4cdd61f3304b9e0900a8c3e699
:*Awaiting feedback if this helps performance of the larger databases.


*Potentially related to mysql indexing since performance gets worse as the number of patients increases. Awaiting further testing/debugging to locate performance bottlenecks before taking any action here.
===MySQL Optimization===
:*Proposed mysql indexes '''('''<span style="color:red;">It has been confirmed that these indexes drastically improve performance of the Patient Summary page and other CDR related page</span>''')
*Partially related to mysql indexing.
:*mysql indexes below have shown to drastically improve performance of the Patient Summary page and other CDR engine related pages:
::*'pid' and 'type' indexes on the 'lists' table
::*'pid' and 'type' indexes on the 'lists' table
::*'pid' index in the 'form_vitals' table
::*'pid' index in the 'form_vitals' table
Line 29: Line 32:
::*'patient_id' index in the 'extended_log' table (stores disclosures)
::*'patient_id' index in the 'extended_log' table (stores disclosures)
::*'patient_id' index in the 'prescriptions' table
::*'patient_id' index in the 'prescriptions' table
:*If add indexes need to also include in the upgrade script and can take advantage of the [http://dev.mysql.com/doc/refman/5.0/en/show-index.html SHOW INDEX] and [http://dev.mysql.com/doc/refman/5.0/en/create-index.html CREATE INDEX] mysql commands to do this in a safe fashion.
:*'''The above indexes have been added to OpenEMR 4.1.1 development version and the database upgrade script for that version. It was also included in the [[OpenEMR Patches|most recent 4.1.0 patch]].'''
 
*Apparently checking redundant diagnosis codes in mysql standard rules. Since it is checked via LIKE %.Value.% then seems like no need to check redundant dx codes (for example, for diabetes ICD9:250 is checked, so seems to be no reason to check the more specific codes ICD9:250.xx). Can probably remove/optimize these all for the diabetes and HTN codes.
 
*Potentially related to coding. Awaiting further testing/debugging to locate performance bottlenecks before taking any action here.
:*Remove php time limit for scripts:
::*openemr/interface/patient_file/reminder/patient_reminders.php
:::*Place '''set_time_limit(0);''' php command at top of script under the require_once stuff
:*Also add a javascript processing animation to the scripts that will take awhile.


===Code Optimization===
*Remove php time limit for scripts:
:*openemr/interface/patient_file/reminder/patient_reminders.php
::*Place '''set_time_limit(0);''' php command at top of script under the require_once stuff
:::*'''The above time limit mod has been added to OpenEMR 4.1.1 development version. It was also included in the [[OpenEMR Patches|most recent 4.1.0 patch]].'''
:*Also plan to add the '''set_time_limit(0);''' to the CDR report script
::*'''The above time limit mod has been added to OpenEMR 4.1.1 development version. It was also included in the [[OpenEMR Patches|most recent 4.1.0 patch]].'''
*Add a non-ajax batching mechanism to decrease memory usage and improve performance. ('''included in 4.1.1 patch''')
*Incorporating session optimizations ( session_write_close() ). ('''included in 4.1.1 patch''')
*Have the CDR engine sql queries bypass the logging engine. ('''included in 4.1.1 patch''')
*Consider optimizing the diagnois filter/target code within the library/clinical_rules.php and CQM/AMC class functions(basically building a middle layer).
:*For example, could aggregate the diagnosis filters query check into one query (rather than doing a query for each check).
*Consider incorporating the above non-ajax batching mechanism into a ajax mechanism which would allow progress monitoring and possibly improve performance (this may not be very helpful since already incorporating non-ajax batching):
:*http://sourceforge.net/projects/openemr/forums/forum/202506/topic/4733322 (post number 45)
:*http://github.com/bradymiller/openemr/commit/54c0bf8844ff2441ae248e37a9f43b9034d8bd36 (prelim code by yehster for patient reminders)


===Testing===
*Debug and reporting from users/developers/vendors with large databases.
*Debug and reporting from users/developers/vendors with large databases.
:*Can either do this via tools(such as Firebug, XDEBUG, KcacheGrind, and/or monitoring sql query freq/time) or via observations and reporting of the following:
:*Can either do this via tools(such as Firebug, XDEBUG, KcacheGrind, and/or monitoring sql query freq/time) or via observations and reporting of the following:
Line 46: Line 59:
::*Testing all the reports in Reports->Clinical to see which ones are running fine and which ones aren't
::*Testing all the reports in Reports->Clinical to see which ones are running fine and which ones aren't
:*Reporting the results in one of the above sourceforge threads or on this wiki page (when reporting, very useful to know the size of the following mysql tables: patient_data, form_encounter, form_vitals, lists).
:*Reporting the results in one of the above sourceforge threads or on this wiki page (when reporting, very useful to know the size of the following mysql tables: patient_data, form_encounter, form_vitals, lists).
[[Category:Clinical Decision Rules]][[Category:Developer Guide]]

Latest revision as of 08:26, 22 September 2012

Issues

Several issues have been brought up in the forums, regarding performance of the CDR engine. Here are the forum threads:


Summary:

  • Patient Summary page widgets related to CDR are slow
  • Notable, this gets slower as the number of patients in the database increases(this has been fixed).
  • Also, the other non-CDR related widgets(with ajax calls) appear to be slowed down by the CDR related widgets(ajax calls).
  • CDR engine based reports are slow
  • So far, the Administration-Reminders reports has been reported to not work with a patient database size of 4500 users. Suspect this issue will also be encountered in other CDR engine based reports. (this has been fixed, albeit the reports take a very long time)

Plan/Fixes

Jquery Optimization

  • Awaiting feedback if this helps performance of the larger databases.

MySQL Optimization

  • Partially related to mysql indexing.
  • mysql indexes below have shown to drastically improve performance of the Patient Summary page and other CDR engine related pages:
  • 'pid' and 'type' indexes on the 'lists' table
  • 'pid' index in the 'form_vitals' table
  • 'pid' index in the 'forms' table
  • 'pid' index in the 'form_encounter' table (only in the upgrade script)
  • 'patient_id' index in the 'immunizations' table
  • 'patient_id' index in the 'procedure_order' table
  • 'pid' index in the 'pnotes' table
  • 'pid' index in the 'transactions' table
  • 'patient_id' index in the 'extended_log' table (stores disclosures)
  • 'patient_id' index in the 'prescriptions' table
  • The above indexes have been added to OpenEMR 4.1.1 development version and the database upgrade script for that version. It was also included in the most recent 4.1.0 patch.
  • Apparently checking redundant diagnosis codes in mysql standard rules. Since it is checked via LIKE %.Value.% then seems like no need to check redundant dx codes (for example, for diabetes ICD9:250 is checked, so seems to be no reason to check the more specific codes ICD9:250.xx). Can probably remove/optimize these all for the diabetes and HTN codes.

Code Optimization

  • Remove php time limit for scripts:
  • openemr/interface/patient_file/reminder/patient_reminders.php
  • Place set_time_limit(0); php command at top of script under the require_once stuff
  • The above time limit mod has been added to OpenEMR 4.1.1 development version. It was also included in the most recent 4.1.0 patch.
  • Also plan to add the set_time_limit(0); to the CDR report script
  • The above time limit mod has been added to OpenEMR 4.1.1 development version. It was also included in the most recent 4.1.0 patch.
  • Add a non-ajax batching mechanism to decrease memory usage and improve performance. (included in 4.1.1 patch)
  • Incorporating session optimizations ( session_write_close() ). (included in 4.1.1 patch)
  • Have the CDR engine sql queries bypass the logging engine. (included in 4.1.1 patch)
  • Consider optimizing the diagnois filter/target code within the library/clinical_rules.php and CQM/AMC class functions(basically building a middle layer).
  • For example, could aggregate the diagnosis filters query check into one query (rather than doing a query for each check).
  • Consider incorporating the above non-ajax batching mechanism into a ajax mechanism which would allow progress monitoring and possibly improve performance (this may not be very helpful since already incorporating non-ajax batching):

Testing

  • Debug and reporting from users/developers/vendors with large databases.
  • Can either do this via tools(such as Firebug, XDEBUG, KcacheGrind, and/or monitoring sql query freq/time) or via observations and reporting of the following:
  • Going through adminstration->alerts and working through the rules one at a time.
  • Set as Patient Reminder to test the Administration->Reminders page or the Patient Reminder widget on the Patient Summary page.
  • Set as Passive Reminders to test the Clinical Reminder widget on the Patient Summary page.
  • Testing all the reports in Reports->Clinical to see which ones are running fine and which ones aren't
  • Reporting the results in one of the above sourceforge threads or on this wiki page (when reporting, very useful to know the size of the following mysql tables: patient_data, form_encounter, form_vitals, lists).