- This work attempts to reveal the comparability issues related to outdoor testing procedures of organic photovoltaic (OPV) modules via studies of inter-laboratory long-term outdoor measurements of roll-to-roll coated flexible OPV modules (P3HT:PCBM, inverted architecture) in different geographic locations from both Southern and Northern hemispheres. The interpretation of the module degradation via sub-cell analyses is presented and the poor reproducibility of the module performance linked to the barrier properties of the encapsulation around the device terminals is addressed. We demonstrate that the modules' t80 lifetime may vary between a few hundred to over 10,000 h depending on how well the device terminals are sealed. We additionally demonstrate up to 17 months of stable performance for sub-cells within the modules. Furthermore, the effects of different geographical locations, weather conditions and measurement setups on the comparability of test results are analyzed. A strong link between the device temperature and performance is revealed, which is ascribed to the reaction of PEDOT:PSS layer with water. The estimation of the true performance of the modules by accommodation of variations in testing conditions is performed. Based on the results a set of recommendations from the ISOS-O guiding protocols are highlighted, which can help remove the factors that affect the comparability of the test results.