![[Jenkins]](/csi/static/a404d988/images/svgs/logo.svg){kind=logo}
Failed
projectroot.test.CHT_NS2d_Scalar2d_FullEmb_FlatSlab_TEST (from CTest)
Failing for the past 2 builds
(Since Failed
)
Stacktrace
OUT/CHT_FlatSlab -------------------------------------------------------------------------- mpirun has exited due to process rank 0 with PID 2527 on node reynolds exiting improperly. There are two reasons this could occur: 1. this process did not call "init" before exiting, but others in the job did. This can cause a job to hang indefinitely while it waits for all processes to call "init". By rule, if one process calls "init", then ALL processes must call "init" prior to termination. 2. this process called "init", but exited without calling "finalize". By rule, all processes that call "init" MUST call "finalize" prior to exiting or it will be considered an "abnormal termination" This may have caused other processes in the application to be terminated by signals sent by mpirun (as reported here). -------------------------------------------------------------------------- > ERROR: ../platforms/release_gnu48/bin/ProjectX exited with error code 247 for CHT_NS2d_Scalar2d_FullEmb_FlatSlab_TEST.job truth = 7.075849433664431842E+01, test = 6.196875272242140653E+01, relerror = 1.242217163695481796E-01 Metric Check mismatch on component 1 of node 68. truth = -7.506853294945969424E+01, test = -6.574337845940000591E+01, relerror = 1.242218826407317789E-01 Metric Check mismatch on component 2 of node 68. truth = 1.452384638730865447E+02, test = 1.271968226344972237E+02, relerror = 1.242208211066912343E-01 Metric Check mismatch on component 0 of node 69. truth = 4.032385859269803063E+01, test = 3.531463737329690389E+01, relerror = 1.242247491738851656E-01 Metric Check mismatch on component 1 of node 69. truth = -3.229320458808154370E+01, test = -2.828140860782714583E+01, relerror = 1.242303460256475034E-01 Metric Check mismatch on component 2 of node 69. truth = 7.214935573672460123E+01, test = 6.318672182906170320E+01, relerror = 1.242233394344898489E-01 Metric Check mismatch on component 0 of node 70. truth = 2.751694626136122324E+01, test = 2.409885380919017805E+01, relerror = 1.242177245870725866E-01 Metric Check mismatch on component 1 of node 70. truth = -1.853366556464891701E+01, test = -1.623161582814375947E+01, relerror = 1.242090901271081238E-01 Metric Check mismatch on component 2 of node 70. truth = 4.200805671838050870E+01, test = 3.678996179944495992E+01, relerror = 1.242165271751879407E-01 Metric Check mismatch on component 0 of node 71. truth = 2.320662150826279202E+01, test = 2.032388851048617440E+01, relerror = 1.242202789729737883E-01 Metric Check mismatch on component 1 of node 71. truth = -1.339042959948020517E+01, test = -1.172709664144831976E+01, relerror = 1.242180428696965239E-01 Metric Check mismatch on component 2 of node 71. truth = 3.125906923735702136E+01, test = 2.737608198603483345E+01, relerror = 1.242195415940829106E-01 Metric Check mismatch on component 0 of node 72. truth = 8.366650620524980653E+00, test = 7.327346598650084530E+00, relerror = 1.242198424451101402E-01 Metric Check mismatch on component 1 of node 72. truth = -3.869003595131863538E-01, test = -3.388423125412983739E-01, relerror = 4.805804697188797991E-02 Metric Check mismatch on component 2 of node 72. truth = 8.460175980090038905E+00, test = 7.409243000587975914E+00, relerror = 1.242211724644146503E-01 Metric Check mismatch on component 0 of node 73. truth = 1.029709639290322443E+01, test = 9.017967632685218504E+00, relerror = 1.242222769808762339E-01 Metric Check mismatch on component 1 of node 73. truth = -4.420399217210248533E-01, test = -3.871366515425885280E-01, relerror = 5.490327017843632529E-02 Metric Check mismatch on component 2 of node 73. truth = 1.055997521559474350E+01, test = 9.248228699673386899E+00, relerror = 1.242187116106293204E-01 Metric Check mismatch on component 0 of node 74. truth = 1.161812795513866270E+01, test = 1.017494619311765369E+01, relerror = 1.242180984400928501E-01 Metric Check mismatch on component 1 of node 74. truth = -1.456271290016669440E+00, test = -1.275375838951766871E+00, relerror = 1.242182361933619611E-01 Metric Check mismatch on component 2 of node 74. truth = 1.392856032504513131E+01, test = 1.219831432998502940E+01, relerror = 1.242228884164663649E-01 Metric Check mismatch on component 0 of node 75. truth = 1.785371707597906976E+01, test = 1.563596887829343807E+01, relerror = 1.242177294648326852E-01 Metric Check mismatch on component 1 of node 75. truth = -8.965041346820447288E+00, test = -7.851406933523561449E+00, relerror = 1.242196628230664845E-01 Metric Check mismatch on component 2 of node 75. truth = 2.687248920025155741E+01, test = 2.353430719480636668E+01, relerror = 1.242230290081925703E-01 Metric Check mismatch on component 0 of node 76. truth = 3.770341940581739948E+01, test = 3.301988008616584835E+01, relerror = 1.242205453367688645E-01 Metric Check mismatch on component 1 of node 76. truth = -3.275826487525576880E+01, test = -2.868901127880150881E+01, relerror = 1.242206695607985384E-01 Metric Check mismatch on component 2 of node 76. truth = 6.794124407200884264E+01, test = 5.950154034291174554E+01, relerror = 1.242206239284066316E-01 Metric Check mismatch on component 0 of node 77. truth = 4.729217076389594610E+01, test = 4.141748405891809170E+01, relerror = 1.242211260359979141E-01 Metric Check mismatch on component 1 of node 77. truth = -4.668890734879736470E+01, test = -4.088914767107353043E+01, relerror = 1.242213623547827051E-01 Metric Check mismatch on component 2 of node 77. truth = 9.715208249305833021E+01, test = 8.508378116268283975E+01, relerror = 1.242207168460623540E-01 Metric Check mismatch on component 0 of node 78. truth = 4.077338396430964451E+01, test = 3.570845877432388704E+01, relerror = 1.242213595618961164E-01 Metric Check mismatch on component 1 of node 78. truth = -3.682785550707534128E+01, test = -3.225305569613419721E+01, relerror = 1.242211838824619219E-01 Metric Check mismatch on component 2 of node 78. truth = 7.808251672726365200E+01, test = 6.838308697940745162E+01, relerror = 1.242202499918845793E-01 Metric Check mismatch on component 0 of node 79. truth = 3.665055637923574494E+01, test = 3.209780638961461818E+01, relerror = 1.242204877468237462E-01 Metric Check mismatch on component 1 of node 79. truth = -2.623950116882685535E+01, test = -2.298004223300155502E+01, relerror = 1.242195465094288603E-01 Metric Check mismatch on component 2 of node 79. truth = 5.260652958443004934E+01, test = 4.607175377264313454E+01, relerror = 1.242198613633888477E-01 Metric Check mismatch on component 0 of node 80. truth = 2.893732170787283664E+01, test = 2.534270357890557790E+01, relerror = 1.242208302916053414E-01 Metric Check mismatch on component 1 of node 80. truth = -1.777616219570509060E+01, test = -1.556797345345368377E+01, relerror = 1.242219056026012541E-01 Metric Check mismatch on component 2 of node 80. truth = 3.812071368770486401E+01, test = 3.338532019270168405E+01, relerror = 1.242210083944596810E-01 Error -9 (Code Flow) has occured. File : /home1/jenkins/workspace/ProjectX_Commit/PX_ARC/release_gnu48/buildnode/reynolds/src/Grid/PXMeshMetric.c Line : 250 Call : PX_CODE_FLOW_ERROR Error -9 (Code Flow) has occured. File : /home1/jenkins/workspace/ProjectX_Commit/PX_ARC/release_gnu48/buildnode/reynolds/src/PXHPAdapt.c Line : 2907 Call : PXCheckTruthMetricFile(pg_meshing, CheckTruthMetricFileRoot, aiter) Error -9 (Code Flow) has occured. File : /home1/jenkins/workspace/ProjectX_Commit/PX_ARC/release_gnu48/buildnode/reynolds/src/PXHPAdapt.c Line : 3321 Call : PXAdjustGridH(pg0,pxa->Parameter,&pgH, aiter, &EfficiencyIndex) Error -9 (Code Flow) has occured. File : /home1/jenkins/workspace/ProjectX_Commit/PX_ARC/release_gnu48/buildnode/reynolds/src/PXHPAdaptDirect.c Line : 8351 Call : PXAdjustGridHP(pxa, pg1,aiter) Error -9 (Code Flow) has occured. File : /home1/jenkins/workspace/ProjectX_Commit/PX_ARC/release_gnu48/buildnode/reynolds/src/PXHPAdapt.c Line : 3717 Call : PXAdaptHPDirect(pxa,aiter,DoneAdapt, &pg1, adaptModeHP) Error -9 (Code Flow) has occured. File : /home1/jenkins/workspace/ProjectX_Commit/PX_ARC/release_gnu48/buildnode/reynolds/src/PXRunSolver.c Line : 790 Call : PXAdaptHP(pxa, aiter, PrimalSolveConvergedFlag, DoneAdapt) Error -9 (Code Flow) has occured. File : /home1/jenkins/workspace/ProjectX_Commit/PX_ARC/release_gnu48/buildnode/reynolds/src/PXRunSolver.c Line : 1026 Call : PXAdaptIteration( pxa, aiter, ConvergedFlag, &DoneAdapt ) 101.811u 0.207s 1:42.03 99.9% 0+0k 0+4336io 0pf+0w Thu Mar 24 14:01:05 EDT 2016 > ERROR: ../platforms/release_gnu48/bin/ProjectX exited with error code 247 for CHT_NS2d_Scalar2d_FullEmb_FlatSlab_TEST.job
Standard Output
OUT/CHT_FlatSlab -------------------------------------------------------------------------- mpirun has exited due to process rank 0 with PID 2527 on node reynolds exiting improperly. There are two reasons this could occur: 1. this process did not call "init" before exiting, but others in the job did. This can cause a job to hang indefinitely while it waits for all processes to call "init". By rule, if one process calls "init", then ALL processes must call "init" prior to termination. 2. this process called "init", but exited without calling "finalize". By rule, all processes that call "init" MUST call "finalize" prior to exiting or it will be considered an "abnormal termination" This may have caused other processes in the application to be terminated by signals sent by mpirun (as reported here). -------------------------------------------------------------------------- > ERROR: ../platforms/release_gnu48/bin/ProjectX exited with error code 247 for CHT_NS2d_Scalar2d_FullEmb_FlatSlab_TEST.job truth = 7.075849433664431842E+01, test = 6.196875272242140653E+01, relerror = 1.242217163695481796E-01 Metric Check mismatch on component 1 of node 68. truth = -7.506853294945969424E+01, test = -6.574337845940000591E+01, relerror = 1.242218826407317789E-01 Metric Check mismatch on component 2 of node 68. truth = 1.452384638730865447E+02, test = 1.271968226344972237E+02, relerror = 1.242208211066912343E-01 Metric Check mismatch on component 0 of node 69. truth = 4.032385859269803063E+01, test = 3.531463737329690389E+01, relerror = 1.242247491738851656E-01 Metric Check mismatch on component 1 of node 69. truth = -3.229320458808154370E+01, test = -2.828140860782714583E+01, relerror = 1.242303460256475034E-01 Metric Check mismatch on component 2 of node 69. truth = 7.214935573672460123E+01, test = 6.318672182906170320E+01, relerror = 1.242233394344898489E-01 Metric Check mismatch on component 0 of node 70. truth = 2.751694626136122324E+01, test = 2.409885380919017805E+01, relerror = 1.242177245870725866E-01 Metric Check mismatch on component 1 of node 70. truth = -1.853366556464891701E+01, test = -1.623161582814375947E+01, relerror = 1.242090901271081238E-01 Metric Check mismatch on component 2 of node 70. truth = 4.200805671838050870E+01, test = 3.678996179944495992E+01, relerror = 1.242165271751879407E-01 Metric Check mismatch on component 0 of node 71. truth = 2.320662150826279202E+01, test = 2.032388851048617440E+01, relerror = 1.242202789729737883E-01 Metric Check mismatch on component 1 of node 71. truth = -1.339042959948020517E+01, test = -1.172709664144831976E+01, relerror = 1.242180428696965239E-01 Metric Check mismatch on component 2 of node 71. truth = 3.125906923735702136E+01, test = 2.737608198603483345E+01, relerror = 1.242195415940829106E-01 Metric Check mismatch on component 0 of node 72. truth = 8.366650620524980653E+00, test = 7.327346598650084530E+00, relerror = 1.242198424451101402E-01 Metric Check mismatch on component 1 of node 72. truth = -3.869003595131863538E-01, test = -3.388423125412983739E-01, relerror = 4.805804697188797991E-02 Metric Check mismatch on component 2 of node 72. truth = 8.460175980090038905E+00, test = 7.409243000587975914E+00, relerror = 1.242211724644146503E-01 Metric Check mismatch on component 0 of node 73. truth = 1.029709639290322443E+01, test = 9.017967632685218504E+00, relerror = 1.242222769808762339E-01 Metric Check mismatch on component 1 of node 73. truth = -4.420399217210248533E-01, test = -3.871366515425885280E-01, relerror = 5.490327017843632529E-02 Metric Check mismatch on component 2 of node 73. truth = 1.055997521559474350E+01, test = 9.248228699673386899E+00, relerror = 1.242187116106293204E-01 Metric Check mismatch on component 0 of node 74. truth = 1.161812795513866270E+01, test = 1.017494619311765369E+01, relerror = 1.242180984400928501E-01 Metric Check mismatch on component 1 of node 74. truth = -1.456271290016669440E+00, test = -1.275375838951766871E+00, relerror = 1.242182361933619611E-01 Metric Check mismatch on component 2 of node 74. truth = 1.392856032504513131E+01, test = 1.219831432998502940E+01, relerror = 1.242228884164663649E-01 Metric Check mismatch on component 0 of node 75. truth = 1.785371707597906976E+01, test = 1.563596887829343807E+01, relerror = 1.242177294648326852E-01 Metric Check mismatch on component 1 of node 75. truth = -8.965041346820447288E+00, test = -7.851406933523561449E+00, relerror = 1.242196628230664845E-01 Metric Check mismatch on component 2 of node 75. truth = 2.687248920025155741E+01, test = 2.353430719480636668E+01, relerror = 1.242230290081925703E-01 Metric Check mismatch on component 0 of node 76. truth = 3.770341940581739948E+01, test = 3.301988008616584835E+01, relerror = 1.242205453367688645E-01 Metric Check mismatch on component 1 of node 76. truth = -3.275826487525576880E+01, test = -2.868901127880150881E+01, relerror = 1.242206695607985384E-01 Metric Check mismatch on component 2 of node 76. truth = 6.794124407200884264E+01, test = 5.950154034291174554E+01, relerror = 1.242206239284066316E-01 Metric Check mismatch on component 0 of node 77. truth = 4.729217076389594610E+01, test = 4.141748405891809170E+01, relerror = 1.242211260359979141E-01 Metric Check mismatch on component 1 of node 77. truth = -4.668890734879736470E+01, test = -4.088914767107353043E+01, relerror = 1.242213623547827051E-01 Metric Check mismatch on component 2 of node 77. truth = 9.715208249305833021E+01, test = 8.508378116268283975E+01, relerror = 1.242207168460623540E-01 Metric Check mismatch on component 0 of node 78. truth = 4.077338396430964451E+01, test = 3.570845877432388704E+01, relerror = 1.242213595618961164E-01 Metric Check mismatch on component 1 of node 78. truth = -3.682785550707534128E+01, test = -3.225305569613419721E+01, relerror = 1.242211838824619219E-01 Metric Check mismatch on component 2 of node 78. truth = 7.808251672726365200E+01, test = 6.838308697940745162E+01, relerror = 1.242202499918845793E-01 Metric Check mismatch on component 0 of node 79. truth = 3.665055637923574494E+01, test = 3.209780638961461818E+01, relerror = 1.242204877468237462E-01 Metric Check mismatch on component 1 of node 79. truth = -2.623950116882685535E+01, test = -2.298004223300155502E+01, relerror = 1.242195465094288603E-01 Metric Check mismatch on component 2 of node 79. truth = 5.260652958443004934E+01, test = 4.607175377264313454E+01, relerror = 1.242198613633888477E-01 Metric Check mismatch on component 0 of node 80. truth = 2.893732170787283664E+01, test = 2.534270357890557790E+01, relerror = 1.242208302916053414E-01 Metric Check mismatch on component 1 of node 80. truth = -1.777616219570509060E+01, test = -1.556797345345368377E+01, relerror = 1.242219056026012541E-01 Metric Check mismatch on component 2 of node 80. truth = 3.812071368770486401E+01, test = 3.338532019270168405E+01, relerror = 1.242210083944596810E-01 Error -9 (Code Flow) has occured. File : /home1/jenkins/workspace/ProjectX_Commit/PX_ARC/release_gnu48/buildnode/reynolds/src/Grid/PXMeshMetric.c Line : 250 Call : PX_CODE_FLOW_ERROR Error -9 (Code Flow) has occured. File : /home1/jenkins/workspace/ProjectX_Commit/PX_ARC/release_gnu48/buildnode/reynolds/src/PXHPAdapt.c Line : 2907 Call : PXCheckTruthMetricFile(pg_meshing, CheckTruthMetricFileRoot, aiter) Error -9 (Code Flow) has occured. File : /home1/jenkins/workspace/ProjectX_Commit/PX_ARC/release_gnu48/buildnode/reynolds/src/PXHPAdapt.c Line : 3321 Call : PXAdjustGridH(pg0,pxa->Parameter,&pgH, aiter, &EfficiencyIndex) Error -9 (Code Flow) has occured. File : /home1/jenkins/workspace/ProjectX_Commit/PX_ARC/release_gnu48/buildnode/reynolds/src/PXHPAdaptDirect.c Line : 8351 Call : PXAdjustGridHP(pxa, pg1,aiter) Error -9 (Code Flow) has occured. File : /home1/jenkins/workspace/ProjectX_Commit/PX_ARC/release_gnu48/buildnode/reynolds/src/PXHPAdapt.c Line : 3717 Call : PXAdaptHPDirect(pxa,aiter,DoneAdapt, &pg1, adaptModeHP) Error -9 (Code Flow) has occured. File : /home1/jenkins/workspace/ProjectX_Commit/PX_ARC/release_gnu48/buildnode/reynolds/src/PXRunSolver.c Line : 790 Call : PXAdaptHP(pxa, aiter, PrimalSolveConvergedFlag, DoneAdapt) Error -9 (Code Flow) has occured. File : /home1/jenkins/workspace/ProjectX_Commit/PX_ARC/release_gnu48/buildnode/reynolds/src/PXRunSolver.c Line : 1026 Call : PXAdaptIteration( pxa, aiter, ConvergedFlag, &DoneAdapt ) 101.811u 0.207s 1:42.03 99.9% 0+0k 0+4336io 0pf+0w Thu Mar 24 14:01:05 EDT 2016 > ERROR: ../platforms/release_gnu48/bin/ProjectX exited with error code 247 for CHT_NS2d_Scalar2d_FullEmb_FlatSlab_TEST.job