v0.9.1
Public Member Functions | Public Attributes | List of all members
MoFEM::CommInterface Struct Reference

Managing BitRefLevels. More...

#include <src/interfaces/CommInterface.hpp>

Inheritance diagram for MoFEM::CommInterface:
[legend]
Collaboration diagram for MoFEM::CommInterface:
[legend]

Public Member Functions

MoFEMErrorCode query_interface (const MOFEMuuid &uuid, UnknownInterface **iface) const
 
 CommInterface (const MoFEM::Core &core)
 
 ~CommInterface ()
 Destructor. More...
 
- Public Member Functions inherited from MoFEM::UnknownInterface
template<class IFACE >
MoFEMErrorCode registerInterface (const MOFEMuuid &uuid, bool error_if_registration_failed=true)
 Register interface. More...
 
template<class IFACE , bool VERIFY = false>
MoFEMErrorCode getInterface (const MOFEMuuid &uuid, IFACE *&iface) const
 Get interface by uuid and return reference to pointer of interface. More...
 
template<class IFACE >
MoFEMErrorCode getInterface (IFACE *&iface) const
 Get interface refernce to pointer of interface. More...
 
template<class IFACE >
MoFEMErrorCode getInterface (IFACE **const iface) const
 Get interface pointer to pointer of interface. More...
 
template<class IFACE , typename boost::enable_if< boost::is_pointer< IFACE >, int >::type = 0>
IFACE getInterface () const
 Get interface pointer to pointer of interface. More...
 
template<class IFACE , typename boost::enable_if< boost::is_reference< IFACE >, int >::type = 0>
IFACE getInterface () const
 Get reference to interface. More...
 
template<class IFACE >
IFACE * getInterface () const
 Function returning pointer to interface. More...
 
virtual ~UnknownInterface ()=default
 
virtual MoFEMErrorCode getLibVersion (Version &version) const
 Get library version. More...
 
virtual const MoFEMErrorCode getFileVersion (moab::Interface &moab, Version &version) const
 Get database major version. More...
 
virtual MoFEMErrorCode getInterfaceVersion (Version &version) const
 Get database major version. More...
 
template<>
MoFEMErrorCode getInterface (const MOFEMuuid &uuid, UnknownInterface *&iface) const
 

Public Attributes

MoFEM::CorecOre
 
bool dEbug
 

Make entities multishared

MoFEMErrorCode makeEntitiesMultishared (const EntityHandle *entities, const int num_entities, const int owner_proc=0, int verb=DEFAULT_VERBOSITY)
 make entities from proc 0 shared on all proc More...
 
MoFEMErrorCode makeEntitiesMultishared (Range &entities, const int owner_proc=0, int verb=DEFAULT_VERBOSITY)
 make entities from proc 0 shared on all proc More...
 
MoFEMErrorCode makeFieldEntitiesMultishared (const std::string field_name, const int owner_proc=0, int verb=DEFAULT_VERBOSITY)
 make field entities multi shared More...
 
MoFEMErrorCode exchangeFieldData (const std::string field_name, int verb=DEFAULT_VERBOSITY)
 Exchange field data. More...
 

Synchronize entities (Following functions in future will be

deprecated)

MoFEMErrorCode synchroniseEntities (Range &ent, int verb=DEFAULT_VERBOSITY)
 
MoFEMErrorCode synchroniseFieldEntities (const std::string name, int verb=DEFAULT_VERBOSITY)
 

Additional Inherited Members

- Protected Member Functions inherited from MoFEM::UnknownInterface
boost::typeindex::type_index getClassIdx (const MOFEMuuid &uid) const
 Get type name for interface Id. More...
 
MOFEMuuid getUId (const boost::typeindex::type_index &class_idx) const
 Get interface Id for class name. More...
 

Detailed Description

Managing BitRefLevels.

Examples
dm_partitioned_no_field.cpp, elasticity.cpp, and mesh_smoothing.cpp.

Definition at line 33 of file CommInterface.hpp.

Constructor & Destructor Documentation

◆ CommInterface()

MoFEM::CommInterface::CommInterface ( const MoFEM::Core core)

Definition at line 34 of file CommInterface.cpp.

35  : cOre(const_cast<MoFEM::Core &>(core)), dEbug(false) {}

◆ ~CommInterface()

MoFEM::CommInterface::~CommInterface ( )

Destructor.

Definition at line 36 of file CommInterface.cpp.

36 {}

Member Function Documentation

◆ exchangeFieldData()

MoFEMErrorCode MoFEM::CommInterface::exchangeFieldData ( const std::string  field_name,
int  verb = DEFAULT_VERBOSITY 
)

Exchange field data.

Exchange field for all shared and ghosted entities. This function should be called collectively over the communicator for this ParallelComm. If the entities vector is empty, all shared entities participate in the exchange. If a proc has no owned entities this function must still be called since it is collective.

Note
collective - need tu be run on all processors in communicator
Todo:
It is not working if field has entities diffrent than vertices.
Parameters
verb
field_name
Returns
MoFEMErrorCode

Definition at line 401 of file CommInterface.cpp.

402  {
403  MoFEM::Interface &m_field = cOre;
405  if (m_field.get_comm_size() > 1) {
406 
407  const FieldEntity_multiIndex *field_ents;
408  CHKERR m_field.get_field_ents(&field_ents);
409 
410  Range exchange_ents_data_verts, exchange_ents_data;
411 
412  for (auto it = field_ents->get<FieldName_mi_tag>().lower_bound(field_name);
413  it != field_ents->get<FieldName_mi_tag>().upper_bound(field_name);
414  ++it)
415  if (
416 
417  ((*it)->getPStatus()) &&
418 
419  (*it)->getNbDofsOnEnt()
420 
421  ) {
422  if ((*it)->getEntType() == MBVERTEX)
423  exchange_ents_data_verts.insert((*it)->getRefEnt());
424  else
425  exchange_ents_data.insert((*it)->getRefEnt());
426  }
427 
428  auto field_ptr = m_field.get_field_structure(field_name);
429  ParallelComm *pcomm = ParallelComm::get_pcomm(
430  &m_field.get_moab(), m_field.get_basic_entity_data_ptr()->pcommID);
431 
432  auto exchange = [&](const Range &ents, Tag th) {
434  if (!ents.empty()) {
435  std::vector<Tag> tags;
436  tags.push_back(th);
437  CHKERR pcomm->exchange_tags(tags, tags, ents);
438  }
440  };
441 
442  CHKERR exchange(exchange_ents_data_verts, field_ptr->th_FieldDataVerts);
443  CHKERR exchange(exchange_ents_data, field_ptr->th_FieldData);
444  }
446 }
Deprecated interface functions.
virtual moab::Interface & get_moab()=0
virtual boost::shared_ptr< BasicEntityData > & get_basic_entity_data_ptr()=0
Get pointer to basic entity data.
virtual int get_comm_size() const =0
multi_index_container< boost::shared_ptr< FieldEntity >, indexed_by< ordered_unique< tag< Unique_mi_tag >, member< FieldEntity, UId, &FieldEntity::globalUId > >, ordered_non_unique< tag< FieldName_mi_tag >, const_mem_fun< FieldEntity::interface_type_Field, boost::string_ref, &FieldEntity::getNameRef > >, ordered_non_unique< tag< Ent_mi_tag >, const_mem_fun< FieldEntity, EntityHandle, &FieldEntity::getEnt > >, ordered_non_unique< tag< Composite_Name_And_Ent_mi_tag >, composite_key< FieldEntity, const_mem_fun< FieldEntity::interface_type_Field, boost::string_ref, &FieldEntity::getNameRef >, const_mem_fun< FieldEntity, EntityHandle, &FieldEntity::getEnt > > > > > FieldEntity_multiIndex
MultiIndex container keeps FieldEntity.
#define MoFEMFunctionReturn(a)
Last executable line of each PETSc function used for error handling. Replaces return()
Definition: definitions.h:482
#define CHKERR
Inline error check.
Definition: definitions.h:601
virtual MoFEMErrorCode get_field_ents(const FieldEntity_multiIndex **field_ents) const =0
Get field multi index.
#define MoFEMFunctionBegin
First executable line of each MoFEM function, used for error handling. Final line of MoFEM functions ...
Definition: definitions.h:412
virtual const Field * get_field_structure(const std::string &name)=0
get field structure

◆ makeEntitiesMultishared() [1/2]

MoFEMErrorCode MoFEM::CommInterface::makeEntitiesMultishared ( const EntityHandle entities,
const int  num_entities,
const int  owner_proc = 0,
int  verb = DEFAULT_VERBOSITY 
)

make entities from proc 0 shared on all proc

Note
collective - need tu be run on all processors in communicator
Parameters
entities
num_entities
my_procdefault proc id to share from
verb
Returns
MoFEMErrorCode

Definition at line 230 of file CommInterface.cpp.

232  {
233  MoFEM::Interface &m_field = cOre;
235 
236  if (m_field.get_comm_size() > 1) {
237 
238  ParallelComm *pcomm = ParallelComm::get_pcomm(
239  &m_field.get_moab(), m_field.get_basic_entity_data_ptr()->pcommID);
240 
241  Range all_ents_range;
242  all_ents_range.insert_list(entities, entities + num_entities);
243 
244  auto get_tag = [&]() {
245  Tag th_gid;
246  const int zero = 0;
247  CHKERR m_field.get_moab().tag_get_handle(
248  "TMP_GLOBAL_ID_TAG_NAME", 1, MB_TYPE_INTEGER, th_gid,
249  MB_TAG_SPARSE | MB_TAG_CREAT, &zero);
250  return th_gid;
251  };
252 
253  auto delete_tag = [&](auto &&th_gid) {
255  CHKERR m_field.get_moab().tag_delete(th_gid);
257  };
258 
259  auto resolve_shared_ents = [&](auto &&th_gid, auto &all_ents_range) {
260  auto set_gid = [&](auto &th_gid) {
261  std::vector<int> gids(num_entities);
262  for (size_t g = 0; g != all_ents_range.size(); ++g)
263  gids[g] = g + 1;
264  CHKERR m_field.get_moab().tag_set_data(th_gid, all_ents_range,
265  &*gids.begin());
266 
267  return &th_gid;
268  };
269 
270  auto get_skin_ents = [&](auto &all_ents_range) {
271  std::array<Range, 4> proc_ents_skin;
272  proc_ents_skin[3] = all_ents_range.subset_by_dimension(3);
273  proc_ents_skin[2] = all_ents_range.subset_by_dimension(2);
274  proc_ents_skin[1] = all_ents_range.subset_by_dimension(1);
275  proc_ents_skin[0] = all_ents_range.subset_by_dimension(0);
276  return proc_ents_skin;
277  };
278 
279  auto resolve_dim = [&](auto &all_ents_range) {
280  for (int resolve_dim = 3; resolve_dim >= 0; --resolve_dim) {
281  if (all_ents_range.num_of_dimension(resolve_dim))
282  return resolve_dim;
283  }
284  return -1;
285  };
286 
287  auto get_proc_ent = [&](auto &all_ents_range) {
288  Range proc_ent;
289  if (m_field.get_comm_rank() == owner_proc)
290  proc_ent = all_ents_range;
291  return proc_ent;
292  };
293 
294  auto resolve_shared_ents = [&](auto &&proc_ents, auto &&skin_ents) {
295  return pcomm->resolve_shared_ents(
296  0, proc_ents, resolve_dim(all_ents_range),
297  resolve_dim(all_ents_range), skin_ents.data(), set_gid(th_gid));
298  };
299 
300  CHKERR resolve_shared_ents(get_proc_ent(all_ents_range),
301  get_skin_ents(all_ents_range));
302 
303  return th_gid;
304  };
305 
306  CHKERR delete_tag(resolve_shared_ents(get_tag(), all_ents_range));
307 
308  if (verb >= NOISY) {
309 
310  auto print_owner = [&](const EntityHandle e) {
312  int moab_owner_proc;
313  EntityHandle moab_owner_handle;
314  CHKERR pcomm->get_owner_handle(e, moab_owner_proc, moab_owner_handle);
315 
316  unsigned char pstatus = 0;
317 
318  CHKERR m_field.get_moab().tag_get_data(pcomm->pstatus_tag(), &e, 1,
319  &pstatus);
320 
321  std::vector<int> shprocs(MAX_SHARING_PROCS, 0);
322  std::vector<EntityHandle> shhandles(MAX_SHARING_PROCS, 0);
323 
324  CHKERR m_field.get_moab().tag_get_data(pcomm->sharedp_tag(), &e, 1,
325  &shprocs[0]);
326  CHKERR m_field.get_moab().tag_get_data(pcomm->sharedh_tag(), &e, 1,
327  &shhandles[0]);
328  if (pstatus & PSTATUS_MULTISHARED) {
329  CHKERR m_field.get_moab().tag_get_data(pcomm->sharedps_tag(), &e, 1,
330  &shprocs[0]);
331  CHKERR m_field.get_moab().tag_get_data(pcomm->sharedhs_tag(), &e, 1,
332  &shhandles[0]);
333  }
334 
335  std::ostringstream ss;
336 
337  ss << "Rank " << m_field.get_comm_rank() << " ";
338  if (!(pstatus & PSTATUS_NOT_OWNED))
339  ss << "OWNER ";
340  if (pstatus & PSTATUS_SHARED)
341  ss << "PSTATUS_SHARED ";
342  if (pstatus & PSTATUS_MULTISHARED)
343  ss << "PSTATUS_MULTISHARED ";
344 
345  ss << "owner " << moab_owner_proc << " (" << owner_proc << ") ";
346 
347  ss << "shprocs: ";
348  for (size_t r = 0; r != m_field.get_comm_size() + 1; ++r)
349  ss << shprocs[r] << " ";
350 
351  ss << "shhandles: ";
352  for (size_t r = 0; r != m_field.get_comm_size() + 1; ++r)
353  ss << shhandles[r] << " ";
354 
355  ss << std::endl;
356  PetscSynchronizedPrintf(m_field.get_comm(), "%s", ss.str().c_str());
357  PetscSynchronizedFlush(m_field.get_comm(), PETSC_STDOUT);
358 
360  };
361 
362  for (auto e : all_ents_range)
363  CHKERR print_owner(e);
364  }
365  }
366 
368 }
Deprecated interface functions.
virtual moab::Interface & get_moab()=0
virtual boost::shared_ptr< BasicEntityData > & get_basic_entity_data_ptr()=0
Get pointer to basic entity data.
virtual int get_comm_size() const =0
#define MoFEMFunctionReturn(a)
Last executable line of each PETSc function used for error handling. Replaces return()
Definition: definitions.h:482
virtual int get_comm_rank() const =0
#define CHKERR
Inline error check.
Definition: definitions.h:601
#define MoFEMFunctionBegin
First executable line of each MoFEM function, used for error handling. Final line of MoFEM functions ...
Definition: definitions.h:412
virtual MPI_Comm & get_comm() const =0

◆ makeEntitiesMultishared() [2/2]

MoFEMErrorCode MoFEM::CommInterface::makeEntitiesMultishared ( Range &  entities,
const int  owner_proc = 0,
int  verb = DEFAULT_VERBOSITY 
)

make entities from proc 0 shared on all proc

Note
collective - need tu be run on all processors in communicator
Parameters
entities
my_procdefault proc id to share from
verb
Returns
MoFEMErrorCode

Definition at line 370 of file CommInterface.cpp.

372  {
373  MoFEM::Interface &m_field = cOre;
375  if (m_field.get_comm_size() > 1) {
376  const int num_ents = entities.size();
377  std::vector<EntityHandle> vec_ents(num_ents);
378  std::copy(entities.begin(), entities.end(), vec_ents.begin());
379  CHKERR makeEntitiesMultishared(&*vec_ents.begin(), num_ents, owner_proc,
380  verb);
381  }
383 }
Deprecated interface functions.
virtual int get_comm_size() const =0
#define MoFEMFunctionReturn(a)
Last executable line of each PETSc function used for error handling. Replaces return()
Definition: definitions.h:482
MoFEMErrorCode makeEntitiesMultishared(const EntityHandle *entities, const int num_entities, const int owner_proc=0, int verb=DEFAULT_VERBOSITY)
make entities from proc 0 shared on all proc
#define CHKERR
Inline error check.
Definition: definitions.h:601
#define MoFEMFunctionBegin
First executable line of each MoFEM function, used for error handling. Final line of MoFEM functions ...
Definition: definitions.h:412

◆ makeFieldEntitiesMultishared()

MoFEMErrorCode MoFEM::CommInterface::makeFieldEntitiesMultishared ( const std::string  field_name,
const int  owner_proc = 0,
int  verb = DEFAULT_VERBOSITY 
)

make field entities multi shared

Note
collective - need tu be run on all processors in communicator
Parameters
field_name
owner_proc
verb
Returns
MoFEMErrorCode

Definition at line 386 of file CommInterface.cpp.

387  {
388  MoFEM::Interface &m_field = cOre;
390  if (m_field.get_comm_size() > 1) {
391  EntityHandle field_meshset = m_field.get_field_meshset(field_name);
392  std::vector<EntityHandle> field_ents;
393  CHKERR m_field.get_moab().get_entities_by_handle(field_meshset, field_ents,
394  true);
395  CHKERR makeEntitiesMultishared(&*field_ents.begin(), field_ents.size(),
396  owner_proc, verb);
397  }
399 }
Deprecated interface functions.
virtual moab::Interface & get_moab()=0
virtual int get_comm_size() const =0
#define MoFEMFunctionReturn(a)
Last executable line of each PETSc function used for error handling. Replaces return()
Definition: definitions.h:482
virtual EntityHandle get_field_meshset(const std::string &name) const =0
get field meshset
MoFEMErrorCode makeEntitiesMultishared(const EntityHandle *entities, const int num_entities, const int owner_proc=0, int verb=DEFAULT_VERBOSITY)
make entities from proc 0 shared on all proc
#define CHKERR
Inline error check.
Definition: definitions.h:601
#define MoFEMFunctionBegin
First executable line of each MoFEM function, used for error handling. Final line of MoFEM functions ...
Definition: definitions.h:412

◆ query_interface()

MoFEMErrorCode MoFEM::CommInterface::query_interface ( const MOFEMuuid uuid,
UnknownInterface **  iface 
) const
virtual

Implements MoFEM::UnknownInterface.

Definition at line 22 of file CommInterface.cpp.

23  {
25  *iface = NULL;
26  if (uuid == IDD_MOFEMComm) {
27  *iface = const_cast<CommInterface *>(this);
29  }
30  SETERRQ(PETSC_COMM_SELF, MOFEM_DATA_INCONSISTENCY, "unknown interface");
32 }
#define MoFEMFunctionBeginHot
First executable line of each MoFEM function, used for error handling. Final line of MoFEM functions ...
Definition: definitions.h:506
#define MoFEMFunctionReturnHot(a)
Last executable line of each PETSc function used for error handling. Replaces return()
Definition: definitions.h:513
static const MOFEMuuid IDD_MOFEMComm

◆ synchroniseEntities()

MoFEMErrorCode MoFEM::CommInterface::synchroniseEntities ( Range &  ent,
int  verb = DEFAULT_VERBOSITY 
)

synchronize entity range on processors (collective)

Note
collective - need tu be run on all processors in communicator

Definition at line 38 of file CommInterface.cpp.

38  {
39  MoFEM::Interface &m_field = cOre;
40  const RefEntity_multiIndex *ref_ents_ptr;
42  CHKERR m_field.get_ref_ents(&ref_ents_ptr);
43 
44  // make a buffer
45  std::vector<std::vector<EntityHandle>> sbuffer(m_field.get_comm_size());
46 
47  Range::iterator eit = ents.begin();
48  for (; eit != ents.end(); eit++) {
49 
50  auto meit = ref_ents_ptr->get<Ent_mi_tag>().find(*eit);
51  if (meit == ref_ents_ptr->get<Ent_mi_tag>().end()) {
52  continue;
53  SETERRQ2(PETSC_COMM_SELF, MOFEM_DATA_INCONSISTENCY,
54  "rank %d entity %lu not exist on database, local entity can not "
55  "be found for this owner",
56  m_field.get_comm_rank(), *eit);
57  }
58 
59  unsigned char pstatus = (*meit)->getPStatus();
60 
61  if (pstatus == 0)
62  continue;
63 
64  if (verb >= NOISY) {
65  std::ostringstream zz;
66  zz << "pstatus " << std::bitset<8>(pstatus) << " ";
67  PetscSynchronizedPrintf(m_field.get_comm(), "%s", zz.str().c_str());
68  }
69 
70  for (int proc = 0;
71  proc < MAX_SHARING_PROCS && -1 != (*meit)->getSharingProcsPtr()[proc];
72  proc++) {
73  if ((*meit)->getSharingProcsPtr()[proc] == -1)
74  SETERRQ(PETSC_COMM_SELF, MOFEM_IMPOSIBLE_CASE,
75  "sharing processor not set");
76 
77  if ((*meit)->getSharingProcsPtr()[proc] == m_field.get_comm_rank())
78  continue;
79 
80  EntityHandle handle_on_sharing_proc =
81  (*meit)->getSharingHandlersPtr()[proc];
82  sbuffer[(*meit)->getSharingProcsPtr()[proc]].push_back(
83  handle_on_sharing_proc);
84  if (verb >= NOISY)
85  PetscSynchronizedPrintf(
86  m_field.get_comm(), "send %lu (%lu) to %d at %d\n",
87  (*meit)->getRefEnt(), handle_on_sharing_proc,
88  (*meit)->getSharingProcsPtr()[proc], m_field.get_comm_rank());
89 
90  if (!(pstatus & PSTATUS_MULTISHARED))
91  break;
92  }
93  }
94 
95  int nsends = 0; // number of messages to send
96  std::vector<int> sbuffer_lengths(
97  m_field.get_comm_size()); // length of the message to proc
98  const size_t block_size = sizeof(EntityHandle) / sizeof(int);
99  for (int proc = 0; proc < m_field.get_comm_size(); proc++) {
100 
101  if (!sbuffer[proc].empty()) {
102 
103  sbuffer_lengths[proc] = sbuffer[proc].size() * block_size;
104  nsends++;
105 
106  } else {
107 
108  sbuffer_lengths[proc] = 0;
109  }
110  }
111 
112  // Make sure it is a PETSc m_field.get_comm()
113  CHKERR PetscCommDuplicate(m_field.get_comm(), &m_field.get_comm(), NULL);
114 
115  std::vector<MPI_Status> status(m_field.get_comm_size());
116 
117  // Computes the number of messages a node expects to receive
118  int nrecvs; // number of messages received
119  CHKERR PetscGatherNumberOfMessages(m_field.get_comm(), NULL,
120  &sbuffer_lengths[0], &nrecvs);
121 
122  // Computes info about messages that a MPI-node will receive, including
123  // (from-id,length) pairs for each message.
124  int *onodes; // list of node-ids from which messages are expected
125  int *olengths; // corresponding message lengths
126  CHKERR PetscGatherMessageLengths(m_field.get_comm(), nsends, nrecvs,
127  &sbuffer_lengths[0], &onodes, &olengths);
128 
129  // Gets a unique new tag from a PETSc communicator. All processors that share
130  // the communicator MUST call this routine EXACTLY the same number of times.
131  // This tag should only be used with the current objects communicator; do NOT
132  // use it with any other MPI communicator.
133  int tag;
134  CHKERR PetscCommGetNewTag(m_field.get_comm(), &tag);
135 
136  // Allocate a buffer sufficient to hold messages of size specified in
137  // olengths. And post Irecvs on these buffers using node info from onodes
138  int **rbuf; // must bee freed by user
139  MPI_Request *r_waits; // must bee freed by user
140  // rbuf has a pointers to messages. It has size of of nrecvs (number of
141  // messages) +1. In the first index a block is allocated,
142  // such that rbuf[i] = rbuf[i-1]+olengths[i-1].
143  CHKERR PetscPostIrecvInt(m_field.get_comm(), tag, nrecvs, onodes, olengths,
144  &rbuf, &r_waits);
145 
146  MPI_Request *s_waits; // status of sens messages
147  CHKERR PetscMalloc1(nsends, &s_waits);
148 
149  // Send messages
150  for (int proc = 0, kk = 0; proc < m_field.get_comm_size(); proc++) {
151  if (!sbuffer_lengths[proc])
152  continue; // no message to send to this proc
153  CHKERR MPI_Isend(&(sbuffer[proc])[0], // buffer to send
154  sbuffer_lengths[proc], // message length
155  MPIU_INT, proc, // to proc
156  tag, m_field.get_comm(), s_waits + kk);
157  kk++;
158  }
159 
160  // Wait for received
161  if (nrecvs)
162  CHKERR MPI_Waitall(nrecvs, r_waits, &status[0]);
163 
164  // Wait for send messages
165  if (nsends)
166  CHKERR MPI_Waitall(nsends, s_waits, &status[0]);
167 
168  if (verb >= VERY_VERBOSE) {
169  PetscSynchronizedPrintf(m_field.get_comm(), "Rank %d nb. before ents %u\n",
170  m_field.get_comm_rank(), ents.size());
171  }
172 
173  // synchronise range
174  for (int kk = 0; kk < nrecvs; kk++) {
175 
176  int len = olengths[kk];
177  int *data_from_proc = rbuf[kk];
178 
179  for (int ee = 0; ee < len; ee += block_size) {
180 
181  EntityHandle ent;
182  bcopy(&data_from_proc[ee], &ent, sizeof(EntityHandle));
183  auto meit = ref_ents_ptr->get<Ent_mi_tag>().find(ent);
184  if (meit == ref_ents_ptr->get<Ent_mi_tag>().end())
185  SETERRQ2(PETSC_COMM_SELF, MOFEM_DATA_INCONSISTENCY,
186  "rank %d entity %lu not exist on database, local entity can "
187  "not be found for this owner",
188  m_field.get_comm_rank(), ent);
189 
190  if (verb >= VERY_VERBOSE)
191  PetscSynchronizedPrintf(
192  m_field.get_comm(), "received %ul (%ul) from %d at %d\n",
193  (*meit)->getRefEnt(), ent, onodes[kk], m_field.get_comm_rank());
194 
195  ents.insert((*meit)->getRefEnt());
196  }
197  }
198 
199  if (verb >= VERBOSE)
200  PetscSynchronizedPrintf(m_field.get_comm(), "Rank %d nb. after ents %u\n",
201  m_field.get_comm_rank(), ents.size());
202 
203  // Cleaning
204  CHKERR PetscFree(s_waits);
205  CHKERR PetscFree(rbuf[0]);
206  CHKERR PetscFree(rbuf);
207  CHKERR PetscFree(r_waits);
208  CHKERR PetscFree(onodes);
209  CHKERR PetscFree(olengths);
210 
211  if (verb >= VERBOSE)
212  PetscSynchronizedFlush(m_field.get_comm(), PETSC_STDOUT);
213 
215 }
virtual MoFEMErrorCode get_ref_ents(const RefEntity_multiIndex **refined_ents_ptr) const =0
Get ref entities multi-index from database.
Deprecated interface functions.
virtual int get_comm_size() const =0
#define MoFEMFunctionReturn(a)
Last executable line of each PETSc function used for error handling. Replaces return()
Definition: definitions.h:482
virtual int get_comm_rank() const =0
#define CHKERR
Inline error check.
Definition: definitions.h:601
multi_index_container< boost::shared_ptr< RefEntity >, indexed_by< ordered_unique< tag< Ent_mi_tag >, member< RefEntity::BasicEntity, EntityHandle, &RefEntity::ent > >, ordered_non_unique< tag< Ent_Ent_mi_tag >, const_mem_fun< RefEntity, EntityHandle, &RefEntity::getParentEnt > >, ordered_non_unique< tag< EntType_mi_tag >, const_mem_fun< RefEntity::BasicEntity, EntityType, &RefEntity::getEntType > >, ordered_non_unique< tag< ParentEntType_mi_tag >, const_mem_fun< RefEntity, EntityType, &RefEntity::getParentEntType > >, ordered_non_unique< tag< Composite_EntType_and_ParentEntType_mi_tag >, composite_key< RefEntity, const_mem_fun< RefEntity::BasicEntity, EntityType, &RefEntity::getEntType >, const_mem_fun< RefEntity, EntityType, &RefEntity::getParentEntType > > >, ordered_non_unique< tag< Composite_ParentEnt_And_EntType_mi_tag >, composite_key< RefEntity, const_mem_fun< RefEntity::BasicEntity, EntityType, &RefEntity::getEntType >, const_mem_fun< RefEntity, EntityHandle, &RefEntity::getParentEnt > > > > > RefEntity_multiIndex
#define MoFEMFunctionBegin
First executable line of each MoFEM function, used for error handling. Final line of MoFEM functions ...
Definition: definitions.h:412
virtual MPI_Comm & get_comm() const =0

Member Data Documentation

◆ cOre

MoFEM::Core& MoFEM::CommInterface::cOre

Definition at line 38 of file CommInterface.hpp.

◆ dEbug

bool MoFEM::CommInterface::dEbug

Definition at line 39 of file CommInterface.hpp.


The documentation for this struct was generated from the following files: