![[Jenkins]](/csi/static/a404d988/images/svgs/logo.svg){kind=logo}
Failed
projectroot.test.CutCell_rae2822_Euler_M0p2_TEST (from CTest)
Failing for the past 2 builds
(Since Failed
)
Stacktrace
OUT/CutCell_rae2822_Euler_M0p2_TEST -------------------------------------------------------------------------- mpirun has exited due to process rank 0 with PID 31876 on node reynolds exiting improperly. There are two reasons this could occur: 1. this process did not call "init" before exiting, but others in the job did. This can cause a job to hang indefinitely while it waits for all processes to call "init". By rule, if one process calls "init", then ALL processes must call "init" prior to termination. 2. this process called "init", but exited without calling "finalize". By rule, all processes that call "init" MUST call "finalize" prior to exiting or it will be considered an "abnormal termination" This may have caused other processes in the application to be terminated by signals sent by mpirun (as reported here). -------------------------------------------------------------------------- > ERROR: ../platforms/release_gnu48/bin/ProjectX exited with error code 247 for CutCell_rae2822_Euler_M0p2_TEST.job Metric Check mismatch on component 2 of node 405. truth = 8.858563818080842367E-01, test = 8.997994934450467275E-01, relerror = 1.394311163696249078E-02 Metric Check mismatch on component 1 of node 406. truth = -2.322898443059347867E-02, test = -1.975704679760774407E-02, relerror = 3.471937632985734601E-03 Metric Check mismatch on component 2 of node 406. truth = 8.045947178389564414E+00, test = 7.984533474064354763E+00, relerror = 7.632874410381339583E-03 Metric Check mismatch on component 2 of node 407. truth = 2.927116343580071955E+01, test = 2.685802453942901735E+01, relerror = 8.244082616204660707E-02 Metric Check mismatch on component 1 of node 408. truth = -5.959801519050599711E-02, test = -7.202670868607130572E-02, relerror = 1.242869349556530861E-02 Metric Check mismatch on component 2 of node 408. truth = 6.291585061848675053E+01, test = 5.936329927079918178E+01, relerror = 5.646512465085725546E-02 Metric Check mismatch on component 1 of node 409. truth = 2.806510001072325955E-01, test = 3.109385273613928802E-01, relerror = 3.028752725416028468E-02 Metric Check mismatch on component 2 of node 409. truth = 1.473544238184612141E+02, test = 1.318434834043985688E+02, relerror = 1.052628079437365749E-01 Metric Check mismatch on component 1 of node 410. truth = -1.516479449600774410E-01, test = -1.098366661733961025E-01, relerror = 4.181127878668133846E-02 Metric Check mismatch on component 2 of node 410. truth = 7.187077841465551842E+01, test = 7.097373863581768205E+01, relerror = 1.248128653431860788E-02 Metric Check mismatch on component 1 of node 411. truth = -1.561253939990412953E-01, test = -1.503136549180751347E-01, relerror = 5.811739080966160653E-03 Metric Check mismatch on component 2 of node 411. truth = 1.465968725280618479E+01, test = 1.427889115399053566E+01, relerror = 2.597573142242556518E-02 Metric Check mismatch on component 1 of node 412. truth = -2.058979963688286133E-02, test = -2.201744400560700851E-02, relerror = 1.427644368724147184E-03 Metric Check mismatch on component 2 of node 412. truth = 4.451868377335442162E+00, test = 4.947104620569151123E+00, relerror = 1.112423372072200850E-01 Metric Check mismatch on component 1 of node 413. truth = 5.729471605871219383E-03, test = 9.496184404271756224E-03, relerror = 3.766712798400536841E-03 Metric Check mismatch on component 2 of node 413. truth = 1.387539683316344252E+00, test = 1.807785072024977735E+00, relerror = 3.028708971437914510E-01 Metric Check mismatch on component 1 of node 414. truth = -8.951548219161017603E-03, test = -1.021986450236420096E-02, relerror = 1.268316283203183362E-03 Metric Check mismatch on component 2 of node 414. truth = 2.864592008921030275E-01, test = 3.733551976007707385E-01, relerror = 8.689599670866771097E-02 Metric Check mismatch on component 1 of node 415. truth = -5.564104215128535050E-03, test = -6.851364773723667145E-03, relerror = 1.287260558595132096E-03 Metric Check mismatch on component 2 of node 415. truth = 4.556483024486221445E-02, test = 5.281603367115265263E-02, relerror = 7.251203426290438181E-03 Metric Check mismatch on component 0 of node 420. truth = 5.655553498518427723E-03, test = 4.554112640759093271E-03, relerror = 1.101440857759334452E-03 Metric Check mismatch on component 2 of node 424. truth = 1.447331710934308263E-02, test = 1.792712017528129420E-02, relerror = 3.453803065938211569E-03 Metric Check mismatch on component 2 of node 425. truth = 2.832336526427121637E-01, test = 2.510631340528745592E-01, relerror = 3.217051858983760448E-02 Metric Check mismatch on component 2 of node 426. truth = 8.200367739616805096E-01, test = 7.045618543303369119E-01, relerror = 1.154749196313435977E-01 Metric Check mismatch on component 2 of node 427. truth = 3.107074310871678513E+00, test = 2.950788804034595980E+00, relerror = 5.029989346899051139E-02 Metric Check mismatch on component 2 of node 428. truth = 2.464811686297441184E+01, test = 2.234540168159753648E+01, relerror = 9.342357447338860144E-02 Metric Check mismatch on component 2 of node 429. truth = 8.371198769293258124E+01, test = 6.946425778156491049E+01, relerror = 1.701993980077305124E-01 Metric Check mismatch on component 1 of node 430. truth = 1.700034159859472316E-01, test = 1.397585067282058890E-01, relerror = 3.024490925774134253E-02 Metric Check mismatch on component 2 of node 430. truth = 2.199941311005264595E+02, test = 1.899102720116350156E+02, relerror = 1.367484620539472784E-01 Metric Check mismatch on component 1 of node 431. truth = -2.937702081939957618E-02, test = -2.756500651357108103E-02, relerror = 1.812014305828495153E-03 Metric Check mismatch on component 2 of node 431. truth = 1.977210970691830312E+02, test = 1.758336076004016775E+02, relerror = 1.106988065169539071E-01 Metric Check mismatch on component 1 of node 432. truth = -1.859836571173645314E-01, test = -2.377289947859185792E-01, relerror = 5.174533766855404782E-02 Metric Check mismatch on component 2 of node 432. truth = 1.792220444655679046E+01, test = 2.444640592900007903E+01, relerror = 3.640289620564347572E-01 Metric Check mismatch on component 1 of node 433. truth = -5.115075592364506235E-02, test = -5.324789080274992120E-02, relerror = 2.097134879104858851E-03 Metric Check mismatch on component 2 of node 433. truth = 5.093637616370471299E+00, test = 5.324591375051140218E+00, relerror = 4.534161557516484808E-02 Metric Check mismatch on component 1 of node 434. truth = -1.853136066535739123E-04, test = -1.675189879641544389E-03, relerror = 1.489876272987970558E-03 Metric Check mismatch on component 2 of node 434. truth = 1.026992023200091309E+00, test = 1.192448192573563936E+00, relerror = 1.611075506291799186E-01 Metric Check mismatch on component 2 of node 435. truth = 3.184176795511218794E-01, test = 4.536090969912044968E-01, relerror = 1.351914174400826174E-01 Metric Check mismatch on component 2 of node 436. truth = 8.982086435436831273E-02, test = 7.517596676098760311E-02, relerror = 1.464489759338070962E-02 Metric Check mismatch on component 2 of node 438. truth = 7.654271037161390970E-03, test = 6.311033029790867437E-03, relerror = 1.343238007370523533E-03 Error -9 (Code Flow) has occured. File : /home1/jenkins/workspace/ProjectX_Commit/PX_ARC/release_gnu48/buildnode/reynolds/src/Grid/PXMeshMetric.c Line : 250 Call : PX_CODE_FLOW_ERROR Error -9 (Code Flow) has occured. File : /home1/jenkins/workspace/ProjectX_Commit/PX_ARC/release_gnu48/buildnode/reynolds/src/PXHPAdapt.c Line : 2907 Call : PXCheckTruthMetricFile(pg_meshing, CheckTruthMetricFileRoot, aiter) Error -9 (Code Flow) has occured. File : /home1/jenkins/workspace/ProjectX_Commit/PX_ARC/release_gnu48/buildnode/reynolds/src/PXHPAdapt.c Line : 3321 Call : PXAdjustGridH(pg0,pxa->Parameter,&pgH, aiter, &EfficiencyIndex) Error -9 (Code Flow) has occured. File : /home1/jenkins/workspace/ProjectX_Commit/PX_ARC/release_gnu48/buildnode/reynolds/src/PXHPAdapt.c Line : 3798 Call : PXAdjustGridHP(pxa, &pg1, aiter) Error -9 (Code Flow) has occured. File : /home1/jenkins/workspace/ProjectX_Commit/PX_ARC/release_gnu48/buildnode/reynolds/src/PXRunSolver.c Line : 790 Call : PXAdaptHP(pxa, aiter, PrimalSolveConvergedFlag, DoneAdapt) Error -9 (Code Flow) has occured. File : /home1/jenkins/workspace/ProjectX_Commit/PX_ARC/release_gnu48/buildnode/reynolds/src/PXRunSolver.c Line : 1026 Call : PXAdaptIteration( pxa, aiter, ConvergedFlag, &DoneAdapt ) 20.553u 0.808s 0:21.41 99.7% 0+0k 0+20264io 0pf+0w Thu Mar 24 13:57:40 EDT 2016 > ERROR: ../platforms/release_gnu48/bin/ProjectX exited with error code 247 for CutCell_rae2822_Euler_M0p2_TEST.job
Standard Output
OUT/CutCell_rae2822_Euler_M0p2_TEST -------------------------------------------------------------------------- mpirun has exited due to process rank 0 with PID 31876 on node reynolds exiting improperly. There are two reasons this could occur: 1. this process did not call "init" before exiting, but others in the job did. This can cause a job to hang indefinitely while it waits for all processes to call "init". By rule, if one process calls "init", then ALL processes must call "init" prior to termination. 2. this process called "init", but exited without calling "finalize". By rule, all processes that call "init" MUST call "finalize" prior to exiting or it will be considered an "abnormal termination" This may have caused other processes in the application to be terminated by signals sent by mpirun (as reported here). -------------------------------------------------------------------------- > ERROR: ../platforms/release_gnu48/bin/ProjectX exited with error code 247 for CutCell_rae2822_Euler_M0p2_TEST.job Metric Check mismatch on component 2 of node 405. truth = 8.858563818080842367E-01, test = 8.997994934450467275E-01, relerror = 1.394311163696249078E-02 Metric Check mismatch on component 1 of node 406. truth = -2.322898443059347867E-02, test = -1.975704679760774407E-02, relerror = 3.471937632985734601E-03 Metric Check mismatch on component 2 of node 406. truth = 8.045947178389564414E+00, test = 7.984533474064354763E+00, relerror = 7.632874410381339583E-03 Metric Check mismatch on component 2 of node 407. truth = 2.927116343580071955E+01, test = 2.685802453942901735E+01, relerror = 8.244082616204660707E-02 Metric Check mismatch on component 1 of node 408. truth = -5.959801519050599711E-02, test = -7.202670868607130572E-02, relerror = 1.242869349556530861E-02 Metric Check mismatch on component 2 of node 408. truth = 6.291585061848675053E+01, test = 5.936329927079918178E+01, relerror = 5.646512465085725546E-02 Metric Check mismatch on component 1 of node 409. truth = 2.806510001072325955E-01, test = 3.109385273613928802E-01, relerror = 3.028752725416028468E-02 Metric Check mismatch on component 2 of node 409. truth = 1.473544238184612141E+02, test = 1.318434834043985688E+02, relerror = 1.052628079437365749E-01 Metric Check mismatch on component 1 of node 410. truth = -1.516479449600774410E-01, test = -1.098366661733961025E-01, relerror = 4.181127878668133846E-02 Metric Check mismatch on component 2 of node 410. truth = 7.187077841465551842E+01, test = 7.097373863581768205E+01, relerror = 1.248128653431860788E-02 Metric Check mismatch on component 1 of node 411. truth = -1.561253939990412953E-01, test = -1.503136549180751347E-01, relerror = 5.811739080966160653E-03 Metric Check mismatch on component 2 of node 411. truth = 1.465968725280618479E+01, test = 1.427889115399053566E+01, relerror = 2.597573142242556518E-02 Metric Check mismatch on component 1 of node 412. truth = -2.058979963688286133E-02, test = -2.201744400560700851E-02, relerror = 1.427644368724147184E-03 Metric Check mismatch on component 2 of node 412. truth = 4.451868377335442162E+00, test = 4.947104620569151123E+00, relerror = 1.112423372072200850E-01 Metric Check mismatch on component 1 of node 413. truth = 5.729471605871219383E-03, test = 9.496184404271756224E-03, relerror = 3.766712798400536841E-03 Metric Check mismatch on component 2 of node 413. truth = 1.387539683316344252E+00, test = 1.807785072024977735E+00, relerror = 3.028708971437914510E-01 Metric Check mismatch on component 1 of node 414. truth = -8.951548219161017603E-03, test = -1.021986450236420096E-02, relerror = 1.268316283203183362E-03 Metric Check mismatch on component 2 of node 414. truth = 2.864592008921030275E-01, test = 3.733551976007707385E-01, relerror = 8.689599670866771097E-02 Metric Check mismatch on component 1 of node 415. truth = -5.564104215128535050E-03, test = -6.851364773723667145E-03, relerror = 1.287260558595132096E-03 Metric Check mismatch on component 2 of node 415. truth = 4.556483024486221445E-02, test = 5.281603367115265263E-02, relerror = 7.251203426290438181E-03 Metric Check mismatch on component 0 of node 420. truth = 5.655553498518427723E-03, test = 4.554112640759093271E-03, relerror = 1.101440857759334452E-03 Metric Check mismatch on component 2 of node 424. truth = 1.447331710934308263E-02, test = 1.792712017528129420E-02, relerror = 3.453803065938211569E-03 Metric Check mismatch on component 2 of node 425. truth = 2.832336526427121637E-01, test = 2.510631340528745592E-01, relerror = 3.217051858983760448E-02 Metric Check mismatch on component 2 of node 426. truth = 8.200367739616805096E-01, test = 7.045618543303369119E-01, relerror = 1.154749196313435977E-01 Metric Check mismatch on component 2 of node 427. truth = 3.107074310871678513E+00, test = 2.950788804034595980E+00, relerror = 5.029989346899051139E-02 Metric Check mismatch on component 2 of node 428. truth = 2.464811686297441184E+01, test = 2.234540168159753648E+01, relerror = 9.342357447338860144E-02 Metric Check mismatch on component 2 of node 429. truth = 8.371198769293258124E+01, test = 6.946425778156491049E+01, relerror = 1.701993980077305124E-01 Metric Check mismatch on component 1 of node 430. truth = 1.700034159859472316E-01, test = 1.397585067282058890E-01, relerror = 3.024490925774134253E-02 Metric Check mismatch on component 2 of node 430. truth = 2.199941311005264595E+02, test = 1.899102720116350156E+02, relerror = 1.367484620539472784E-01 Metric Check mismatch on component 1 of node 431. truth = -2.937702081939957618E-02, test = -2.756500651357108103E-02, relerror = 1.812014305828495153E-03 Metric Check mismatch on component 2 of node 431. truth = 1.977210970691830312E+02, test = 1.758336076004016775E+02, relerror = 1.106988065169539071E-01 Metric Check mismatch on component 1 of node 432. truth = -1.859836571173645314E-01, test = -2.377289947859185792E-01, relerror = 5.174533766855404782E-02 Metric Check mismatch on component 2 of node 432. truth = 1.792220444655679046E+01, test = 2.444640592900007903E+01, relerror = 3.640289620564347572E-01 Metric Check mismatch on component 1 of node 433. truth = -5.115075592364506235E-02, test = -5.324789080274992120E-02, relerror = 2.097134879104858851E-03 Metric Check mismatch on component 2 of node 433. truth = 5.093637616370471299E+00, test = 5.324591375051140218E+00, relerror = 4.534161557516484808E-02 Metric Check mismatch on component 1 of node 434. truth = -1.853136066535739123E-04, test = -1.675189879641544389E-03, relerror = 1.489876272987970558E-03 Metric Check mismatch on component 2 of node 434. truth = 1.026992023200091309E+00, test = 1.192448192573563936E+00, relerror = 1.611075506291799186E-01 Metric Check mismatch on component 2 of node 435. truth = 3.184176795511218794E-01, test = 4.536090969912044968E-01, relerror = 1.351914174400826174E-01 Metric Check mismatch on component 2 of node 436. truth = 8.982086435436831273E-02, test = 7.517596676098760311E-02, relerror = 1.464489759338070962E-02 Metric Check mismatch on component 2 of node 438. truth = 7.654271037161390970E-03, test = 6.311033029790867437E-03, relerror = 1.343238007370523533E-03 Error -9 (Code Flow) has occured. File : /home1/jenkins/workspace/ProjectX_Commit/PX_ARC/release_gnu48/buildnode/reynolds/src/Grid/PXMeshMetric.c Line : 250 Call : PX_CODE_FLOW_ERROR Error -9 (Code Flow) has occured. File : /home1/jenkins/workspace/ProjectX_Commit/PX_ARC/release_gnu48/buildnode/reynolds/src/PXHPAdapt.c Line : 2907 Call : PXCheckTruthMetricFile(pg_meshing, CheckTruthMetricFileRoot, aiter) Error -9 (Code Flow) has occured. File : /home1/jenkins/workspace/ProjectX_Commit/PX_ARC/release_gnu48/buildnode/reynolds/src/PXHPAdapt.c Line : 3321 Call : PXAdjustGridH(pg0,pxa->Parameter,&pgH, aiter, &EfficiencyIndex) Error -9 (Code Flow) has occured. File : /home1/jenkins/workspace/ProjectX_Commit/PX_ARC/release_gnu48/buildnode/reynolds/src/PXHPAdapt.c Line : 3798 Call : PXAdjustGridHP(pxa, &pg1, aiter) Error -9 (Code Flow) has occured. File : /home1/jenkins/workspace/ProjectX_Commit/PX_ARC/release_gnu48/buildnode/reynolds/src/PXRunSolver.c Line : 790 Call : PXAdaptHP(pxa, aiter, PrimalSolveConvergedFlag, DoneAdapt) Error -9 (Code Flow) has occured. File : /home1/jenkins/workspace/ProjectX_Commit/PX_ARC/release_gnu48/buildnode/reynolds/src/PXRunSolver.c Line : 1026 Call : PXAdaptIteration( pxa, aiter, ConvergedFlag, &DoneAdapt ) 20.553u 0.808s 0:21.41 99.7% 0+0k 0+20264io 0pf+0w Thu Mar 24 13:57:40 EDT 2016 > ERROR: ../platforms/release_gnu48/bin/ProjectX exited with error code 247 for CutCell_rae2822_Euler_M0p2_TEST.job