v0.16.3
Loading...
Searching...
No Matches
mofem/atom_tests/rebuild_problem.cpp

Recreate a problem on uniformly refined levels while the previous level's problem is alive, and compare it with a fresh build.

Recreate a problem on uniformly refined levels while the previous level's problem is alive, and compare it with a fresh build.Level 0 is built fresh. For every further level l the mesh of level l-1 is refined uniformly to level l, then:

  1. problem P<l>_rebuild is built on bit l with mask {l-1, l} (every bit from the first local refinement on, whose unrefined elements keep their older bits) while the problem of level l-1 is alive: the recreate path of doc/development/rebuild_problems.md;
  2. the level l-1 problem is checked to be unchanged and destroyed;
  3. problem P<l> is built fresh on bit l with the default mask;
  4. the two are compared: DOF and element counts, every DOF's global and local index and part, absence of DOFs on source-only parents, and the matrix size, non-zeros and every owned row's column indices;
  5. the fresh problem is destroyed and the rebuilt one becomes the source of the next level.

Every stage is a TestingInterface test (number 10*l + stage) registered with addTest and run in order by runTests; the stages share one state and each checks that the previous one has run. Phase times are PETSc log events RP_Refine, RP_FieldsFEs, RP_Rebuild, RP_Fresh and RP_Matrix; run with -log_view.

Options: -file_name, -nb_levels, -order; -dm_is_partitioned uses the distributed builder (one rank, or a partitioned mesh whose partition sets are updated after refinement, which this test does not do); -incremental (default true) registers only the entities each refinement creates, through build_fields(ents), build_finite_elements(name, &ents) and build_adjacencies(ents), false uses the all-entities overloads; -local_from <l> refines only one vertex star from level l on, so the cost of a local change on a fixed mesh can be measured; -skip_matrix omits the matrix graphs; -in_place (with -dm_is_partitioned) rebuilds the alive problem under its own name and DM with DMMoFEMRebuildProblem instead of stage 1, and checks the survivor index sets it returns: every pair maps an old global index to the new one of the same DOF, every surviving owned DOF is listed, and a VecScatter through them carries values.

/** \file rebuild_problem.cpp
* \example mofem/atom_tests/rebuild_problem.cpp
* \brief Recreate a problem on uniformly refined levels while the previous
* level's problem is alive, and compare it with a fresh build.
*
* Level 0 is built fresh. For every further level l the mesh of level l-1 is
* refined uniformly to level l, then:
*
* 1. problem P<l>_rebuild is built on bit l with mask {l-1, l} (every bit
* from the first local refinement on, whose unrefined elements keep
* their older bits) while the problem of level l-1 is alive: the
* recreate path of doc/development/rebuild_problems.md;
* 2. the level l-1 problem is checked to be unchanged and destroyed;
* 3. problem P<l> is built fresh on bit l with the default mask;
* 4. the two are compared: DOF and element counts, every DOF's global and
* local index and part, absence of DOFs on source-only parents, and the
* matrix size, non-zeros and every owned row's column indices;
* 5. the fresh problem is destroyed and the rebuilt one becomes the source
* of the next level.
*
* Every stage is a TestingInterface test (number 10*l + stage) registered
* with addTest and run in order by runTests; the stages share one state and
* each checks that the previous one has run. Phase times are PETSc log
* events RP_Refine, RP_FieldsFEs, RP_Rebuild, RP_Fresh and RP_Matrix; run
* with -log_view.
*
* Options: -file_name, -nb_levels, -order; -dm_is_partitioned uses the
* distributed builder (one rank, or a partitioned mesh whose partition sets
* are updated after refinement, which this test does not do); -incremental
* (default true) registers only the entities each refinement creates,
* through build_fields(ents), build_finite_elements(name, &ents) and
* build_adjacencies(ents), false uses the all-entities overloads;
* -local_from <l> refines only one vertex star from level l on, so the cost
* of a local change on a fixed mesh can be measured; -skip_matrix omits the
* matrix graphs; -in_place (with -dm_is_partitioned) rebuilds the alive
* problem under its own name and DM with DMMoFEMRebuildProblem instead of
* stage 1, and checks the survivor index sets it returns: every pair maps
* an old global index to the new one of the same DOF, every surviving owned
* DOF is listed, and a VecScatter through them carries values.
*/
#include <MoFEM.hpp>
#include <map>
#include <boost/functional/hash.hpp>
using namespace MoFEM;
static char help[] =
"Build a problem on uniformly refined levels while the previous level's "
"problem is alive, and compare with a fresh build.\n"
"-file_name <mesh> -nb_levels <n> -order <p> [-dm_is_partitioned]\n\n";
namespace {
constexpr const char *fieldName = "F1";
constexpr const char *elementName = "E1";
inline BitRefLevel bit(int l) { return BitRefLevel().set(l); }
inline BitRefLevel allBits() { return BitRefLevel().set(); }
struct Events {
PetscLogEvent refine, fieldsFes, rebuild, fresh, matrix;
};
/// Counts and a hash of the numbering of one problem, in UID order.
struct Summary {
int globalDofs = 0, localDofs = 0, ghostDofs = 0, elements = 0;
std::size_t numbering = 0;
};
/// Differences between the rebuilt and the fresh problem of one level.
struct Mismatch {
int missing = 0; ///< Rebuilt DOFs absent from the fresh problem.
int index = 0; ///< DOFs with a different global/local index or part.
int parents = 0; ///< Rebuilt DOFs on entities without the level bit.
};
/// Keep only the rebuilt matrix's local graph while the fresh matrix is made.
struct MatrixGraph {
PetscInt firstRow = 0, lastRow = 0, differentRows = 0;
std::vector<std::size_t> offsets;
std::vector<PetscInt> columns;
};
/// Add the entities of level l to the field and the element, and build.
/// Incrementally: only entities created by the refinement (on bit l but not
/// on bit l-1) are added, ordered and built, with the range overloads
/// build_fields(ents), build_finite_elements(name, &ents) and
/// build_adjacencies(ents). Otherwise the all-entities overloads are used,
/// which revisit every entity of every level.
MoFEMErrorCode registerLevel(MoFEM::Interface &m_field, int l, int order,
bool incremental) {
auto bit_mng = m_field.getInterface<BitRefManager>();
Range ents;
CHKERR bit_mng->getEntitiesByRefLevel(bit(l), allBits(), ents);
if (incremental && l > 0) {
CHKERR bit_mng->getEntitiesByRefLevel(bit(l - 1), allBits(), previous);
ents = subtract(ents, previous);
}
for (auto type : {MBVERTEX, MBEDGE, MBTRI, MBTET}) {
Range of_type = ents.subset_by_type(type);
CHKERR m_field.add_ents_to_field_by_type(of_type, type, fieldName);
CHKERR m_field.set_field_order(of_type, fieldName,
type == MBVERTEX ? 1 : order);
}
elementName, MBTET);
if (incremental) {
const Range tets = ents.subset_by_type(MBTET);
CHKERR m_field.build_fields(ents);
CHKERR m_field.build_finite_elements(elementName, &tets);
CHKERR m_field.build_adjacencies(tets);
} else {
CHKERR m_field.build_fields();
}
}
/// Entities listed more than once in the data views of all finite elements.
MoFEMErrorCode countDuplicateViewEntries(MoFEM::Interface &m_field,
int &duplicates) {
duplicates = 0;
std::vector<UId> uids;
for (auto &fe : *m_field.get_ents_finite_elements()) {
auto &view = *fe->getDataFieldEntsPtr();
uids.clear();
for (auto &weak : view)
if (auto ent = weak.lock())
uids.push_back(ent->getLocalUniqueId());
std::sort(uids.begin(), uids.end());
duplicates += uids.size() - (std::unique(uids.begin(), uids.end()) -
uids.begin());
}
}
/// Refine level l-1 onto level l: every tetrahedron, or, when local, only
/// the star of the lowest vertex of the level, with the unchanged
/// tetrahedra and their closure also given bit l, as a crack step leaves
/// unchanged entities on the output level. Returns the number refined.
MoFEMErrorCode refineLevel(MoFEM::Interface &m_field, int l, bool local,
int &refined) {
auto bit_mng = m_field.getInterface<BitRefManager>();
auto &moab = m_field.get_moab();
Range level_tets;
CHKERR bit_mng->getEntitiesByDimAndRefLevel(bit(l - 1), allBits(), 3,
level_tets);
Range tets = level_tets;
if (local) {
Range verts;
CHKERR moab.get_connectivity(level_tets, verts, true);
const EntityHandle vertex = verts.front();
Range star;
CHKERR moab.get_adjacencies(&vertex, 1, 3, false, star);
tets = intersect(star, level_tets);
}
refined = tets.size();
Range edges;
CHKERR moab.get_adjacencies(tets, 1, true, edges, moab::Interface::UNION);
auto refine = m_field.getInterface<MeshRefinement>();
CHKERR refine->refineTets(tets, bit(l));
if (local) {
const Range unchanged = subtract(level_tets, tets);
CHKERR bit_mng->setBitRefLevel(unchanged, bit(l), true);
}
}
/// Check the survivor index sets of an in-place rebuild: every pair maps
/// the old global index of a DOF to its new one, every surviving owned DOF
/// is listed, and a VecScatter through the two sets carries values from a
/// vector of the old layout to one of the new.
MoFEMErrorCode checkSurvivors(MoFEM::Interface &m_field,
const std::string &tag, const std::string &name,
const std::map<UId, int> &old_global,
int old_local, int old_total,
const auto zero = TestingInterface::Tolerance{0, 0};
const Problem *prb;
CHKERR m_field.get_problem(name, &prb);
PetscInt n_old, n_new;
CHKERR ISGetLocalSize(is_old, &n_old);
CHKERR ISGetLocalSize(is_new, &n_new);
CHKERR testing.checkValue<TestType::SCALAR>(
tag + " survivor sets have one size", n_old, n_new, zero);
const PetscInt *old_idx, *new_idx;
CHKERR ISGetIndices(is_old, &old_idx);
CHKERR ISGetIndices(is_new, &new_idx);
auto &by_global = prb->getNumeredRowDofsPtr()->get<PetscGlobalIdx_mi_tag>();
int wrong = 0;
for (PetscInt i = 0; i != n_new; ++i) {
auto it = by_global.find(new_idx[i]);
if (it == by_global.end()) {
++wrong;
continue;
}
auto o = old_global.find((*it)->getLocalUniqueId());
if (o == old_global.end() || o->second != old_idx[i])
++wrong;
}
CHKERR testing.checkValue<TestType::SCALAR>(
tag + " survivor pairs map old to new index", wrong, 0, zero);
int expected = 0;
auto &by_local = prb->getNumeredRowDofsPtr()->get<PetscLocalIdx_mi_tag>();
for (auto it = by_local.lower_bound(0),
hi = by_local.lower_bound(prb->getNbLocalDofsRow());
it != hi; ++it)
if (old_global.count((*it)->getLocalUniqueId()))
++expected;
CHKERR testing.checkValue<TestType::SCALAR>(
tag + " every surviving owned dof is listed", n_new, expected, zero);
// A vector of the old layout holding its own global index, scattered into
// a vector of the new layout through the two sets.
Vec old_vec, new_vec;
CHKERR VecCreateMPI(m_field.get_comm(), old_local, old_total, &old_vec);
PetscInt lo, hi;
CHKERR VecGetOwnershipRange(old_vec, &lo, &hi);
{
PetscScalar *a;
CHKERR VecGetArray(old_vec, &a);
for (PetscInt i = lo; i != hi; ++i)
a[i - lo] = i;
CHKERR VecRestoreArray(old_vec, &a);
}
CHKERR VecCreateMPI(m_field.get_comm(), prb->getNbLocalDofsRow(),
prb->getNbDofsRow(), &new_vec);
CHKERR VecSet(new_vec, -1);
VecScatter scatter;
CHKERR VecScatterCreate(old_vec, is_old, new_vec, is_new, &scatter);
CHKERR VecScatterBegin(scatter, old_vec, new_vec, INSERT_VALUES,
SCATTER_FORWARD);
CHKERR VecScatterEnd(scatter, old_vec, new_vec, INSERT_VALUES,
SCATTER_FORWARD);
CHKERR VecScatterDestroy(&scatter);
PetscInt new_lo, new_hi;
CHKERR VecGetOwnershipRange(new_vec, &new_lo, &new_hi);
int wrong_scatter = 0;
{
const PetscScalar *a;
CHKERR VecGetArrayRead(new_vec, &a);
for (PetscInt i = 0; i != n_new; ++i) {
if (new_idx[i] < new_lo || new_idx[i] >= new_hi)
++wrong_scatter;
else if (a[new_idx[i] - new_lo] != static_cast<PetscScalar>(old_idx[i]))
++wrong_scatter;
}
CHKERR VecRestoreArrayRead(new_vec, &a);
}
CHKERR testing.checkValue<TestType::SCALAR>(
tag + " scatter through survivors carries values", wrong_scatter, 0,
zero);
CHKERR VecDestroy(&old_vec);
CHKERR VecDestroy(&new_vec);
CHKERR ISRestoreIndices(is_old, &old_idx);
CHKERR ISRestoreIndices(is_new, &new_idx);
}
/// Create a DM for a new problem name on the given level and mask and set it
/// up; the DM owns the problem.
MoFEMErrorCode buildProblem(MoFEM::Interface &m_field, const std::string &name,
BitRefLevel level, BitRefLevel mask,
PetscBool is_partitioned, PetscLogEvent event,
SmartPetscObj<DM> &dm, double &seconds) {
if (m_field.check_problem(name))
SETERRQ(PETSC_COMM_SELF, MOFEM_DATA_INCONSISTENCY,
"Problem %s exists: its previous owner was not destroyed",
name.c_str());
dm = createDM(m_field.get_comm(), "DMMOFEM");
CHKERR DMMoFEMCreateMoFEM(dm, &m_field, name.c_str(), level, mask);
// DMMoFEMCreateMoFEM ORs the mask onto the all-ones default of a new
// problem, so the mask given above is not in force; set the selector.
CHKERR DMMoFEMSetDestroyProblem(dm, PETSC_TRUE);
CHKERR DMMoFEMSetIsPartitioned(dm, is_partitioned);
CHKERR DMMoFEMSetSquareProblem(dm, PETSC_TRUE);
CHKERR DMMoFEMAddElement(dm, elementName);
PetscLogDouble t0, t1;
CHKERR PetscTime(&t0);
PetscLogEventBegin(event, 0, 0, 0, 0);
CHKERR DMSetUp(dm);
PetscLogEventEnd(event, 0, 0, 0, 0);
CHKERR PetscTime(&t1);
seconds = t1 - t0;
}
MoFEMErrorCode summarise(MoFEM::Interface &m_field, const std::string &name,
Summary &s) {
const Problem *prb;
CHKERR m_field.get_problem(name, &prb);
s.globalDofs = prb->getNbDofsRow();
s.localDofs = prb->getNbLocalDofsRow();
s.ghostDofs = prb->getNbGhostDofsRow();
s.elements = prb->getNumeredFiniteElementsPtr()->size();
s.numbering = 0;
for (auto &dof : *prb->getNumeredRowDofsPtr()) {
const UId uid = dof->getLocalUniqueId();
boost::hash_combine(s.numbering, static_cast<unsigned long long>(uid));
boost::hash_combine(s.numbering,
static_cast<unsigned long long>(uid >> 64));
boost::hash_combine(s.numbering, dof->getPetscGlobalDofIdx());
boost::hash_combine(s.numbering, dof->getPetscLocalDofIdx());
boost::hash_combine(s.numbering, dof->getPart());
}
}
MoFEMErrorCode compareProblems(MoFEM::Interface &m_field,
const std::string &rebuilt,
const std::string &fresh, BitRefLevel level,
Mismatch &m) {
const Problem *prb_rebuilt, *prb_fresh;
CHKERR m_field.get_problem(rebuilt, &prb_rebuilt);
CHKERR m_field.get_problem(fresh, &prb_fresh);
auto &fresh_by_uid =
prb_fresh->getNumeredRowDofsPtr()->get<Unique_mi_tag>();
for (auto &dof : *prb_rebuilt->getNumeredRowDofsPtr()) {
if (!(dof->getBitRefLevel() & level).any())
++m.parents;
auto it = fresh_by_uid.find(dof->getLocalUniqueId());
if (it == fresh_by_uid.end()) {
++m.missing;
continue;
}
if ((*it)->getPetscGlobalDofIdx() != dof->getPetscGlobalDofIdx() ||
(*it)->getPetscLocalDofIdx() != dof->getPetscLocalDofIdx() ||
(*it)->getPart() != dof->getPart())
++m.index;
}
}
/// Assemble the sparsity of the problem's matrix and report its size.
MoFEMErrorCode matrixInfo(MoFEM::Interface &m_field, const std::string &name,
PetscLogEvent event, double &rows, double &nonzeros,
double &seconds, MatrixGraph &graph,
bool compare = false) {
PetscLogDouble t0, t1;
CHKERR PetscTime(&t0);
PetscLogEventBegin(event, 0, 0, 0, 0);
->createMPIAIJWithArrays<PetscGlobalIdx_mi_tag>(name, A);
PetscLogEventEnd(event, 0, 0, 0, 0);
CHKERR PetscTime(&t1);
seconds = t1 - t0;
PetscInt m, n;
CHKERR MatGetSize(A, &m, &n);
MatInfo info;
CHKERR MatGetInfo(A, MAT_GLOBAL_SUM, &info);
rows = m;
nonzeros = info.nz_used;
// Graph validation is outside the matrix creation timer. Keep one CSR
// snapshot, rather than retaining two full PETSc matrices at large orders.
PetscInt first_row, last_row;
CHKERR MatGetOwnershipRange(A, &first_row, &last_row);
if (compare) {
if (first_row != graph.firstRow || last_row != graph.lastRow)
SETERRQ(m_field.get_comm(), MOFEM_DATA_INCONSISTENCY,
"Matrix ownership ranges differ");
} else {
graph.firstRow = first_row;
graph.lastRow = last_row;
graph.differentRows = 0;
graph.offsets = {0};
graph.offsets.reserve(last_row - first_row + 1);
graph.columns.clear();
MatInfo local_info;
CHKERR MatGetInfo(A, MAT_LOCAL, &local_info);
graph.columns.reserve(static_cast<std::size_t>(local_info.nz_used));
}
std::vector<PetscInt> row_columns;
for (PetscInt row = first_row; row < last_row; ++row) {
PetscInt nb_cols;
const PetscInt *cols;
CHKERR MatGetRow(A, row, &nb_cols, &cols, nullptr);
row_columns.clear();
if (nb_cols)
row_columns.assign(cols, cols + nb_cols);
CHKERR MatRestoreRow(A, row, &nb_cols, &cols, nullptr);
std::sort(row_columns.begin(), row_columns.end());
if (compare) {
const auto local_row = row - first_row;
const auto begin = graph.offsets[local_row];
const auto end = graph.offsets[local_row + 1];
if (row_columns.size() != end - begin ||
!std::equal(row_columns.begin(), row_columns.end(),
graph.columns.begin() + begin))
++graph.differentRows;
} else {
graph.columns.insert(graph.columns.end(), row_columns.begin(),
row_columns.end());
graph.offsets.push_back(graph.columns.size());
}
}
}
/// State shared by the registered stages of one run.
struct State {
SmartPetscObj<DM> alive, rebuilt, fresh;
std::string aliveName, rebuiltName, freshName;
Summary aliveSummary, rebuiltSummary, freshSummary;
double aliveSeconds = 0, rebuiltSeconds = 0, freshSeconds = 0;
int level = -1; ///< Last level whose mesh and fields are registered.
};
} // namespace
int main(int argc, char *argv[]) {
MoFEM::Core::Initialize(&argc, &argv, (char *)0, help);
try {
moab::Core mb_instance;
moab::Interface &moab = mb_instance;
char mesh_file_name[255];
PetscBool flg = PETSC_FALSE;
CHKERR PetscOptionsGetString(PETSC_NULLPTR, "", "-file_name",
mesh_file_name, 255, &flg);
if (flg != PETSC_TRUE)
SETERRQ(PETSC_COMM_SELF, MOFEM_NOT_FOUND,
"-file_name <mesh> is required");
PetscInt nb_levels = 2, order = 3;
PetscBool is_partitioned = PETSC_FALSE, skip_matrix = PETSC_FALSE;
CHKERR PetscOptionsGetInt(PETSC_NULLPTR, "", "-nb_levels", &nb_levels,
PETSC_NULLPTR);
CHKERR PetscOptionsGetInt(PETSC_NULLPTR, "", "-order", &order,
PETSC_NULLPTR);
CHKERR PetscOptionsGetBool(PETSC_NULLPTR, "", "-dm_is_partitioned",
&is_partitioned, PETSC_NULLPTR);
// Skip the matrix graphs: at large sizes they cost more memory than the
// problems themselves and are not part of the build being measured.
CHKERR PetscOptionsGetBool(PETSC_NULLPTR, "", "-skip_matrix", &skip_matrix,
PETSC_NULLPTR);
// Register each level's new entities only (default), or rebuild every
// field entity and element with the all-entities overloads.
PetscBool incremental = PETSC_TRUE;
CHKERR PetscOptionsGetBool(PETSC_NULLPTR, "", "-incremental", &incremental,
PETSC_NULLPTR);
// From this level on, refine only the star of one vertex instead of the
// whole mesh, so registration and build costs can be compared for a
// local change on a mesh of fixed size. Negative: always uniform.
PetscInt local_from = -1;
CHKERR PetscOptionsGetInt(PETSC_NULLPTR, "", "-local_from", &local_from,
PETSC_NULLPTR);
// Rebuild the alive problem in place (DMMoFEMRebuildProblem, same name
// and DM) instead of building the output under a second name; needs the
// distributed builder.
PetscBool in_place = PETSC_FALSE;
CHKERR PetscOptionsGetBool(PETSC_NULLPTR, "", "-in_place", &in_place,
PETSC_NULLPTR);
if (in_place && !is_partitioned)
SETERRQ(PETSC_COMM_SELF, MOFEM_INVALID_DATA,
"-in_place needs -dm_is_partitioned");
CHKERR moab.load_file(mesh_file_name, 0, "");
MoFEM::Core core(moab);
MoFEM::Interface &m_field = core;
CHKERR DMRegister_MoFEM("DMMOFEM");
Events ev;
CHKERR PetscLogEventRegister("RP_Refine", 0, &ev.refine);
CHKERR PetscLogEventRegister("RP_FieldsFEs", 0, &ev.fieldsFes);
CHKERR PetscLogEventRegister("RP_Rebuild", 0, &ev.rebuild);
CHKERR PetscLogEventRegister("RP_Fresh", 0, &ev.fresh);
CHKERR PetscLogEventRegister("RP_Matrix", 0, &ev.matrix);
MOFEM_LOG_TAG("WORLD", "RebuildProblem");
MOFEM_LOG("WORLD", Sev::inform)
<< "Levels " << nb_levels << ", order " << order << ", "
<< (is_partitioned ? "distributed" : "replicated") << " builder, "
<< (incremental ? "incremental" : "full")
<< " field/element registration, "
<< (in_place ? "in-place rebuild" : "recreate path");
auto count_tets = [&](int l) {
Range tets;
bit(l), allBits(), 3, tets),
"count tets");
return tets.size();
};
auto log_level = [&](int l, const std::string &name, const Summary &s,
double seconds) {
MOFEM_LOG_TAG("WORLD", "RebuildProblem");
MOFEM_LOG("WORLD", Sev::inform)
<< "Level " << l << " " << name << ": tets " << count_tets(l)
<< ", elements " << s.elements << ", dofs " << s.globalDofs
<< " (local " << s.localDofs << ", ghost " << s.ghostDofs
<< "), build " << seconds << " s";
};
auto zero = TestingInterface::Tolerance{0, 0};
State st;
// Mask of the rebuilt problem on level l. After a uniform refinement the
// level's elements carry bit l only, and {l-1, l} selects them; a local
// refinement adds bit l to the unrefined elements, which keep their
// older bits, so from the first local level on every bit is admitted.
auto rebuilt_mask = [&](int l) {
return (local_from >= 0 && l >= local_from) ? allBits()
: (bit(l - 1) | bit(l));
};
// Fields and the element are declared once; entities join per level.
CHKERR m_field.getInterface<BitRefManager>()->setBitRefLevelByDim(
0, 3, bit(0));
CHKERR m_field.add_field(fieldName, H1, AINSWORTH_LEGENDRE_BASE, 3,
MB_TAG_DENSE);
CHKERR m_field.add_finite_element(elementName);
CHKERR m_field.modify_finite_element_add_field_row(elementName, fieldName);
CHKERR m_field.modify_finite_element_add_field_col(elementName, fieldName);
fieldName);
auto require = [&](bool ok, const std::string &what) {
if (!ok)
SETERRQ(PETSC_COMM_SELF, MOFEM_DATA_INCONSISTENCY,
"Stage order broken: %s", what.c_str());
};
// Stage 1 of level 0: register the mesh and build the first problem.
CHKERR testing->addTest(1, "level 0 build", [&]() {
PetscLogEventBegin(ev.fieldsFes, 0, 0, 0, 0);
CHKERR registerLevel(m_field, 0, order, incremental);
PetscLogEventEnd(ev.fieldsFes, 0, 0, 0, 0);
st.level = 0;
st.aliveName = "P0";
CHKERR buildProblem(m_field, st.aliveName, bit(0), allBits(),
is_partitioned, ev.fresh, st.alive, st.aliveSeconds);
CHKERR summarise(m_field, st.aliveName, st.aliveSummary);
log_level(0, st.aliveName, st.aliveSummary, st.aliveSeconds);
CHKERR testing->checkValue<TestType::SCALAR>(
"level 0 has dofs", st.aliveSummary.globalDofs > 0 ? 1 : 0, 1,
zero);
});
for (int l = 1; l <= nb_levels; ++l) {
const auto tag = "level " + std::to_string(l);
const int base = 10 * l;
CHKERR testing->addTest(base + 1, tag + " refine", [&, l, tag]() {
CHKERR require(st.level == l - 1 && st.alive,
tag + " refine needs level " + std::to_string(l - 1) +
" registered and its problem alive");
const bool local = local_from >= 0 && l >= local_from;
const auto tets_before = count_tets(l - 1);
int refined;
PetscLogEventBegin(ev.refine, 0, 0, 0, 0);
CHKERR refineLevel(m_field, l, local, refined);
PetscLogEventEnd(ev.refine, 0, 0, 0, 0);
PetscLogDouble t0, t1;
CHKERR PetscTime(&t0);
PetscLogEventBegin(ev.fieldsFes, 0, 0, 0, 0);
CHKERR registerLevel(m_field, l, order, incremental);
PetscLogEventEnd(ev.fieldsFes, 0, 0, 0, 0);
CHKERR PetscTime(&t1);
st.level = l;
int duplicates;
CHKERR countDuplicateViewEntries(m_field, duplicates);
MOFEM_LOG_TAG("WORLD", "RebuildProblem");
MOFEM_LOG("WORLD", Sev::inform)
<< "Level " << l << (local ? " local" : " uniform")
<< " refinement of " << refined
<< " tets; fields and elements registered in " << t1 - t0
<< " s, duplicate view entries " << duplicates;
// Each refined tetrahedron is replaced by eight.
CHKERR testing->checkValue<TestType::SCALAR>(
tag + " tets after refinement", count_tets(l),
tets_before + 7. * refined, zero);
CHKERR testing->checkValue<TestType::SCALAR>(
tag + " element views without duplicates", duplicates, 0, zero);
});
// Recreate path: the output problem is built while the source is
// alive, the source is checked and then destroyed. In-place path: the
// alive problem is rebuilt under its own name and DM, and the survivor
// index sets are checked.
CHKERR testing->addTest(base + 2, tag + " rebuild", [&, l, tag]() {
CHKERR require(st.level == l && st.alive && !st.rebuilt,
tag + " rebuild needs the refined level and the "
"previous problem alive");
if (in_place) {
std::map<UId, int> old_global;
{
const Problem *prb;
CHKERR m_field.get_problem(st.aliveName, &prb);
for (auto &dof : *prb->getNumeredRowDofsPtr())
if (dof->getPetscGlobalDofIdx() >= 0)
old_global[dof->getLocalUniqueId()] =
dof->getPetscGlobalDofIdx();
}
IS raw_old, raw_new;
PetscLogDouble t0, t1;
CHKERR PetscTime(&t0);
PetscLogEventBegin(ev.rebuild, 0, 0, 0, 0);
CHKERR DMMoFEMRebuildProblem(st.alive, bit(l), rebuilt_mask(l),
&raw_old, &raw_new);
PetscLogEventEnd(ev.rebuild, 0, 0, 0, 0);
CHKERR PetscTime(&t1);
SmartPetscObj<IS> is_old(raw_old), is_new(raw_new);
st.rebuilt = st.alive;
st.alive.reset();
st.rebuiltName = st.aliveName;
st.rebuiltSeconds = t1 - t0;
CHKERR summarise(m_field, st.rebuiltName, st.rebuiltSummary);
log_level(l, st.rebuiltName + " in place", st.rebuiltSummary,
st.rebuiltSeconds);
CHKERR testing->checkValue<TestType::SCALAR>(
tag + " problem keeps its name",
m_field.check_problem(st.rebuiltName) ? 1 : 0, 1, zero);
CHKERR testing->checkValue<TestType::SCALAR>(
tag + " staging problem removed",
m_field.check_problem(st.rebuiltName + "__REBUILD_STAGE") ? 1
: 0,
0, zero);
CHKERR checkSurvivors(m_field, *testing, tag, st.rebuiltName,
old_global, st.aliveSummary.localDofs,
st.aliveSummary.globalDofs, is_old, is_new);
}
st.rebuiltName = "P" + std::to_string(l) + "_rebuild";
CHKERR buildProblem(m_field, st.rebuiltName, bit(l), rebuilt_mask(l),
is_partitioned, ev.rebuild, st.rebuilt,
st.rebuiltSeconds);
CHKERR summarise(m_field, st.rebuiltName, st.rebuiltSummary);
log_level(l, st.rebuiltName, st.rebuiltSummary, st.rebuiltSeconds);
Summary source_after;
CHKERR summarise(m_field, st.aliveName, source_after);
CHKERR testing->checkValue<TestType::SCALAR>(
tag + " source dofs unchanged", source_after.globalDofs,
st.aliveSummary.globalDofs, zero);
CHKERR testing->checkValue<TestType::SCALAR>(
tag + " source numbering unchanged",
source_after.numbering == st.aliveSummary.numbering ? 0 : 1, 0,
zero);
// Commit: destroying the source DM deletes its problem.
st.alive.reset();
CHKERR testing->checkValue<TestType::SCALAR>(
tag + " source problem deleted",
m_field.check_problem(st.aliveName) ? 1 : 0, 0, zero);
});
// Oracle: the same level built fresh, alone.
CHKERR testing->addTest(base + 3, tag + " fresh build", [&, l, tag]() {
CHKERR require(st.level == l && st.rebuilt && !st.alive && !st.fresh,
tag + " fresh build needs the rebuilt problem and "
"the source destroyed");
st.freshName = "P" + std::to_string(l);
CHKERR buildProblem(m_field, st.freshName, bit(l), allBits(),
is_partitioned, ev.fresh, st.fresh,
st.freshSeconds);
CHKERR summarise(m_field, st.freshName, st.freshSummary);
log_level(l, st.freshName, st.freshSummary, st.freshSeconds);
});
CHKERR testing->addTest(base + 4, tag + " compare", [&, l, tag]() {
CHKERR require(st.level == l && st.rebuilt && st.fresh,
tag + " compare needs both problems");
const auto &r = st.rebuiltSummary, &f = st.freshSummary;
CHKERR testing->checkValue<TestType::SCALAR>(
tag + " global dofs", r.globalDofs, f.globalDofs, zero);
CHKERR testing->checkValue<TestType::SCALAR>(
tag + " local dofs", r.localDofs, f.localDofs, zero);
CHKERR testing->checkValue<TestType::SCALAR>(
tag + " ghost dofs", r.ghostDofs, f.ghostDofs, zero);
CHKERR testing->checkValue<TestType::SCALAR>(
tag + " elements", r.elements, f.elements, zero);
CHKERR testing->checkValue<TestType::SCALAR>(
tag + " numbering hash equal",
r.numbering == f.numbering ? 0 : 1, 0, zero);
Mismatch mismatch;
CHKERR compareProblems(m_field, st.rebuiltName, st.freshName, bit(l),
mismatch);
CHKERR testing->checkValue<TestType::SCALAR>(
tag + " dofs missing from fresh", mismatch.missing, 0, zero);
CHKERR testing->checkValue<TestType::SCALAR>(
tag + " dofs with different index", mismatch.index, 0, zero);
CHKERR testing->checkValue<TestType::SCALAR>(
tag + " dofs on source-only parents", mismatch.parents, 0, zero);
});
// Matrix graphs agree; then the fresh oracle goes and the rebuilt
// problem is the next source.
CHKERR testing->addTest(base + 5, tag + " matrix", [&, l, tag]() {
CHKERR require(st.level == l && st.rebuilt && st.fresh,
tag + " matrix needs both problems");
if (!skip_matrix) {
double rows_rebuilt, nnz_rebuilt, rows_fresh, nnz_fresh;
double seconds_rebuilt, seconds_fresh;
MatrixGraph graph;
CHKERR matrixInfo(m_field, st.rebuiltName, ev.matrix, rows_rebuilt,
nnz_rebuilt, seconds_rebuilt, graph);
CHKERR matrixInfo(m_field, st.freshName, ev.matrix, rows_fresh,
nnz_fresh, seconds_fresh, graph, true);
MOFEM_LOG_TAG("WORLD", "RebuildProblem");
MOFEM_LOG("WORLD", Sev::inform)
<< "Level " << l << " matrix rows " << rows_rebuilt
<< ", non-zeros " << nnz_rebuilt << ", created in "
<< seconds_rebuilt << " s and " << seconds_fresh << " s";
CHKERR testing->checkValue<TestType::SCALAR>(
tag + " matrix rows", rows_rebuilt, rows_fresh, zero);
CHKERR testing->checkValue<TestType::SCALAR>(
tag + " matrix rows equal dofs", rows_rebuilt,
st.rebuiltSummary.globalDofs, zero);
CHKERR testing->checkValue<TestType::SCALAR>(
tag + " matrix non-zeros", nnz_rebuilt, nnz_fresh, zero);
CHKERR testing->checkValue<TestType::SCALAR>(
tag + " matrix rows with different column indices",
graph.differentRows, 0, zero);
}
st.fresh.reset();
st.alive = st.rebuilt;
st.rebuilt.reset();
st.aliveName = st.rebuiltName;
st.aliveSummary = st.rebuiltSummary;
st.aliveSeconds = st.rebuiltSeconds;
});
}
// Stages run in ascending number: level 0, then refine, rebuild, fresh,
// compare and matrix for each level.
CHKERR testing->runTests();
}
return 0;
}
std::map< EntityHandle, int > index
WorkMesh base
std::vector< char > alive
PetscLogEvent event
Tag tag
std::map< EntityHandle, size_t > offsets
std::map< EntityHandle, std::pair< bool, std::vector< double > > > previous
std::string type
std::vector< FieldBitNumber > row
Definition Schur.cpp:2488
It it
Definition Schur.cpp:2568
It end
Definition Schur.cpp:2568
SmartPetscObj< IS > rows
std::string name
static char help[]
int main()
constexpr double a
#define CATCH_ERRORS
Catch errors.
@ AINSWORTH_LEGENDRE_BASE
Ainsworth Cole (Legendre) approx. base .
Definition definitions.h:60
#define CHK_THROW_MESSAGE(err, msg)
Check and throw MoFEM exception.
#define MoFEMFunctionReturnHot(a)
Last executable line of each PETSc function used for error handling. Replaces return()
@ H1
continuous field
Definition definitions.h:85
#define MoFEMFunctionBegin
First executable line of each MoFEM function, used for error handling. Final line of MoFEM functions ...
@ MOFEM_NOT_FOUND
Definition definitions.h:33
@ MOFEM_DATA_INCONSISTENCY
Definition definitions.h:31
@ MOFEM_INVALID_DATA
Definition definitions.h:36
#define MoFEMFunctionReturn(a)
Last executable line of each PETSc function used for error handling. Replaces return()
#define CHKERR
Inline error check.
constexpr int order
PetscErrorCode DMMoFEMRebuildProblem(DM dm, const MoFEM::BitRefLevel target_bit, const MoFEM::BitRefLevel target_mask, IS *old_row_indices=PETSC_NULLPTR, IS *new_row_indices=PETSC_NULLPTR)
Rebuild the DM's parent problem in place on a new bit ref level.
Definition DMMoFEM.cpp:479
virtual const Problem * get_problem(const std::string problem_name) const =0
Get the problem object.
virtual const EntFiniteElement_multiIndex * get_ents_finite_elements() const =0
Get the ents finite elements object.
MoFEMErrorCode getEntitiesByDimAndRefLevel(const BitRefLevel bit, const BitRefLevel mask, const int dim, const EntityHandle meshset, int verb=0) const
add all ents from ref level given by bit to meshset
virtual MoFEMErrorCode add_finite_element(const std::string &fe_name, enum MoFEMTypes bh=MF_EXCL, int verb=DEFAULT_VERBOSITY)=0
add finite element
virtual MoFEMErrorCode build_finite_elements(int verb=DEFAULT_VERBOSITY)=0
Build finite elements.
virtual MoFEMErrorCode modify_finite_element_add_field_col(const std::string &fe_name, const std::string name_row)=0
set field col which finite element use
virtual MoFEMErrorCode modify_finite_element_add_field_row(const std::string &fe_name, const std::string name_row)=0
set field row which finite element use
virtual MoFEMErrorCode add_ents_to_finite_element_by_bit_ref(const BitRefLevel bit, const BitRefLevel mask, const std::string name, EntityType type, int verb=DEFAULT_VERBOSITY)=0
add TET entities from given refinement level to finite element database given by name
virtual MoFEMErrorCode modify_finite_element_add_field_data(const std::string &fe_name, const std::string name_field)=0
set finite element field data
virtual MoFEMErrorCode build_fields(int verb=DEFAULT_VERBOSITY)=0
virtual MoFEMErrorCode set_field_order(const EntityHandle meshset, const EntityType type, const std::string &name, const ApproximationOrder order, int verb=DEFAULT_VERBOSITY)=0
Set order approximation of the entities in the field.
virtual MoFEMErrorCode add_ents_to_field_by_type(const Range &ents, const EntityType type, const std::string &name, int verb=DEFAULT_VERBOSITY)=0
Add entities to field meshset.
#define MOFEM_LOG(channel, severity)
Log.
#define MOFEM_LOG_TAG(channel, tag)
Tag channel.
#define MOFEM_LOG_CHANNEL(channel)
Set and reset channel.
virtual MoFEMErrorCode modify_problem_mask_ref_level_set_bit(const std::string &name_problem, const BitRefLevel &bit)=0
set dof mask ref level for problem
virtual MoFEMErrorCode modify_problem_ref_level_set_bit(const std::string &name_problem, const BitRefLevel &bit)=0
set ref level for problem
virtual bool check_problem(const std::string name)=0
check if problem exist
auto bit
set bit
FTensor::Index< 'i', SPACE_DIM > i
PetscBool testing
constexpr int nb_levels
Definition level_set.cpp:58
FTensor::Index< 'l', 3 > l
PetscErrorCode MoFEMErrorCode
MoFEM/PETSc error code.
uint128_t UId
Unique Id.
Definition Types.hpp:31
std::bitset< BITREFLEVEL_SIZE > BitRefLevel
Bit structure attached to each entity identifying to what mesh entity is attached.
Definition Types.hpp:40
implementation of Data Operators for Forces and Sources
Definition Common.hpp:10
int r
Definition sdf.py:205
constexpr AssemblyType A
FTensor::Index< 'm', 3 > m
Managing BitRefLevels.
virtual moab::Interface & get_moab()=0
virtual MoFEMErrorCode build_adjacencies(const Range &ents, int verb=DEFAULT_VERBOSITY)=0
build adjacencies
virtual MoFEMErrorCode add_field(const std::string name, const FieldSpace space, const FieldApproximationBase base, const FieldCoefficientsNumber nb_of_coefficients, const TagType tag_type=MB_TAG_SPARSE, const enum MoFEMTypes bh=MF_EXCL, int verb=DEFAULT_VERBOSITY)=0
Add field.
virtual MPI_Comm & get_comm() const =0
Core (interface) class.
Definition Core.hpp:83
static MoFEMErrorCode Initialize(int *argc, char ***args, const char file[], const char help[])
Initializes the MoFEM database PETSc, MOAB and MPI.
Definition Core.cpp:68
static MoFEMErrorCode Finalize()
Checks for options to be called at the conclusion of the program.
Definition Core.cpp:123
Deprecated interface functions.
Matrix manager is used to build and partition problems.
Mesh refinement interface.
MoFEMErrorCode addVerticesInTheMiddleOfEdges(const EntityHandle meshset, const BitRefLevel &bit, const bool recursive=false, int verb=QUIET, EntityHandle start_v=0)
make vertices in the middle of edges in meshset and add them to refinement levels defined by bit
Index tags of the numbered DOF containers that also read the index.
keeps basic data about problem
DofIdx getNbLocalDofsRow() const
DofIdx getNbDofsRow() const
const auto & getNumeredFiniteElementsPtr() const
get access to reference for multi-index storing finite elements
DofIdx getNbGhostDofsRow() const
auto & getNumeredRowDofsPtr() const
get access to numeredRowDofsPtr storing DOFs on rows
intrusive_ptr for managing petsc objects
MoFEMErrorCode getInterface(IFACE *&iface) const
Get interface reference to pointer of interface.