Point Cloud AIS shape

// here is a code for displaying point clouds

// ive been, getting good results, i can load 40 million points max before running out of memory on 32 bit
// and 60 million points on 4 bit before running out of memory, whats funny is i still have memory available i have 8 gigs of memory, im not expert in memory, but i am suspecting, the memory manager stablishes a limit on memory consumtion, even when the operating system has the available memory.
or the task manager doesnt is not accurat enough to know how much memory is actually being used.
// but all in all i get the visual studio out of memory error after the 40mil-60mil points thresholds.

// here is the code, sorry for the use of Qt for point list's, can be easily made without the qlist, but im a newbie, not ready to let go of my qt training wheels.

// best

// AlexP

/// header file :

// AIS_PointCloud.h: interface for the AIS_PointCloud class.
//
//////////////////////////////////////////////////////////////////////

#if !defined(AFX_AIS_PointCloud_H__A9B277C4_A69E_11D1_8DA4_0800369C8A03__INCLUDED_)
#define AFX_AIS_PointCloud_H__A9B277C4_A69E_11D1_8DA4_0800369C8A03__INCLUDED_

#if _MSC_VER >= 1000
#pragma once
#endif // _MSC_VER >= 1000
#include
#include

#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
//#include
#include
#include
#include
#include
#include

class TCollection_AsciiString;
class PrsMgr_PresentationManager2d;
class Graphic2d_GraphicObject;
class SelectMgr_Selection;

DEFINE_STANDARD_HANDLE(AIS_PointCloud,AIS_InteractiveObject)
class AIS_PointCloud : public AIS_InteractiveObject
{
public:
//AIS_PointCloud(const Graphic3d_Array1OfVertex &points);
AIS_PointCloud(QList vertlist);

virtual ~AIS_PointCloud();

DEFINE_STANDARD_RTTI(AIS_PointCloud)

protected:

// Methods PROTECTED
//

// Fields PROTECTED
//

private:

// Methods PRIVATE
//

void Compute (const Handle(PrsMgr_PresentationManager3d)& aPresentationManager,
const Handle(Prs3d_Presentation)& aPresentation,
const Standard_Integer aMode);
void Compute (const Handle(Prs3d_Projector)& aProjector,
const Handle(Prs3d_Presentation)& aPresentation);
void Compute (const Handle(PrsMgr_PresentationManager2d)& aPresentationManager,
const Handle(Graphic2d_GraphicObject)& aGrObj,
const Standard_Integer unMode = 0) ;
void ComputeSelection (const Handle(SelectMgr_Selection)& aSelection,
const Standard_Integer unMode) ;

// Fields PRIVATE
//
//const Graphic3d_Array1OfVertex &cloud_data;
QList vertlist;

};

#endif // !defined(AFX_AIS_PointCloud_H__A9B277C4_A69E_11D1_8DA4_0800369C8A03__INCLUDED_)

// AIS_PointCloud.cpp: implementation of the AIS_PointCloud class.
//
//////////////////////////////////////////////////////////////////////

//// following the code on this link:
//http://www.opencascade.org/org/forum/thread_1125/

#include "AIS_PointCloud.hxx"

#ifdef _DEBUG
#undef THIS_FILE
static char THIS_FILE[]=__FILE__;
//#define new DEBUG_NEW
#endif
IMPLEMENT_STANDARD_HANDLE(AIS_PointCloud,AIS_InteractiveObject)
IMPLEMENT_STANDARD_RTTI(AIS_PointCloud,AIS_InteractiveObject)
//
// Foreach ancestors, we add a IMPLEMENT_STANDARD_SUPERTYPE and
// a IMPLEMENT_STANDARD_SUPERTYPE_ARRAY_ENTRY macro.
// We must respect the order: from the direct ancestor class
// to the base class.
//
IMPLEMENT_STANDARD_TYPE(AIS_PointCloud)
IMPLEMENT_STANDARD_SUPERTYPE(AIS_InteractiveObject)
IMPLEMENT_STANDARD_SUPERTYPE(SelectMgr_SelectableObject)
IMPLEMENT_STANDARD_SUPERTYPE(PrsMgr_PresentableObject)
IMPLEMENT_STANDARD_SUPERTYPE(MMgt_TShared)
IMPLEMENT_STANDARD_SUPERTYPE(Standard_Transient)
IMPLEMENT_STANDARD_SUPERTYPE_ARRAY()
IMPLEMENT_STANDARD_SUPERTYPE_ARRAY_ENTRY(AIS_InteractiveObject)
IMPLEMENT_STANDARD_SUPERTYPE_ARRAY_ENTRY(SelectMgr_SelectableObject)
IMPLEMENT_STANDARD_SUPERTYPE_ARRAY_ENTRY(PrsMgr_PresentableObject)
IMPLEMENT_STANDARD_SUPERTYPE_ARRAY_ENTRY(MMgt_TShared)
IMPLEMENT_STANDARD_SUPERTYPE_ARRAY_ENTRY(Standard_Transient)
IMPLEMENT_STANDARD_SUPERTYPE_ARRAY_END()
IMPLEMENT_STANDARD_TYPE_END(AIS_PointCloud)

#include
#include
#include
#include
#include
#include
#include "PrsMgr_PresentationManager2d.hxx"
#include "SelectMgr_Selection.hxx"
#include
#include
#include
#include
#include
#include

//////////////////////////////////////////////////////////////////////
// Construction/Destruction
//////////////////////////////////////////////////////////////////////

//AIS_PointCloud::AIS_PointCloud(const Graphic3d_Array1OfVertex &points)
//:cloud_data(points)
//{
//
//}

AIS_PointCloud::AIS_PointCloud(QList vlist)
:vertlist(vlist)
{

}

AIS_PointCloud::~AIS_PointCloud()
{

}

void AIS_PointCloud::Compute(const Handle(PrsMgr_PresentationManager3d)& aPresentationManager,
const Handle(Prs3d_Presentation)& aPresentation,
const Standard_Integer aMode)
{

Handle(Graphic3d_Structure) theStructure = Handle(Graphic3d_Structure)::DownCast(aPresentation);

Handle(Graphic3d_Group) theGroup= new Graphic3d_Group(theStructure);

double pcount = vertlist.length();
Handle_Graphic3d_ArrayOfPoints thepointcloud2= new Graphic3d_ArrayOfPoints(pcount);

int index =0;

index =0;
for(int i=0 ; i {
index++;
thepointcloud2->AddVertex(vertlist.at(i));
//thepointcloud2->SetVertexColor(vertindx,colorlist.at(i));
}

theGroup->AddPrimitiveArray(thepointcloud2);

}

void AIS_PointCloud::Compute(const Handle(Prs3d_Projector)& aProjector,
const Handle(Prs3d_Presentation)& aPresentation)
{
}

void AIS_PointCloud::Compute(const Handle(PrsMgr_PresentationManager2d)& aPresentationManager,
const Handle(Graphic2d_GraphicObject)& aGrObj,
const Standard_Integer unMode)
{
}

void AIS_PointCloud::ComputeSelection(const Handle(SelectMgr_Selection)& aSelection,
const Standard_Integer unMode)
{

}

Attachments: 
AP's picture

Dear Forum Supervisor,

Would be nice if you guys could do tests on trying to get the largest amount of points shown using this code and maybe let us know of your results.

I did some direct opengl tests to load more than 300 million points, Animated! and it works smoothly on the same machine, the occ code above is working on static points, so there should be a way of displaying billions of points with opengl, the beginprimitive array is already calling opengl, so i wonder where is the memory growing, in Qt or OCC or Opengl.

// this could also just be a trivial problem directly linked to my novice experience in dealing with memory.

please let me know if you think there is any use in trying to pursue this route.

Best,

AlexP

jelle's picture

> I did some direct opengl tests to load more than 300 million points, Animated!

i think that's already pretty impressive, given OCC's aging OpenGL calls [ which is more like nineties OGL 2 than 3 / 4 idioms, using vertexbuffers ]. so good job!!! and again, thanks for sharing...
have you considered opening up a repository of your code on github?
that's a much more suitable place to share code...

-jelle

AP's picture

Thanks Jelle,

Yes ive considered the GitHub/Google Code/sourceforge, and eventually will, i just find it hard to manage an opensource project while managing a thesis, but it may be a sub-component of my thesis to make an opensource project. i might just make an unsupported project repository where i dump code, and afterwards ill maintain it or hand it over.

In regards to the OpenGl stuff, have you done any work with the userdraw call back mechanism, i think the old voxel in 6.3 forced the user to have the userdraw, they now ship a voxel example in 6.4+

it would be interesting to use the userdraw to call opengl directly for drawing Large count of primitives, in theory this is what the NIS objects are supposed to do, haven had luck with properly using them, will give it a go instead of reinventing the wheel.

And yes ill look into the repository idea, soon, :)

Best,

AlexP

jelle's picture

> i just find it hard to manage an opensource project while managing a thesis, but it may be a sub-component of my thesis to make an opensource project

i disagree with you here, i think creating science & open source software simultaneously is *the* way to go!
condense your knowledge and be able to share it... there is just so much inactionable research since its hard to "package" it.
not so if you work with OCC ;)

many of the arguments why are covered well by Gael Varoquaux:
http://gael-varoquaux.info/blog/?p=144

AP's picture

Thanks for the Blog, really good read.

Ill eventually do it, jeje, dont want to perish either, jeje.

Best,

AlexP

Forum supervisor's picture

Dear Alex,
Could you take care to provide full self-contained test with "main" procedure (if you ask to test something).
Besides you should take into account that when using custom allocators (OCCT in your case), the memory is hoarded by the allocator for reusing purpose in the future and will not be released till the end of application.
Also to accelerate performance you may use UserDraw mode (see Graphic3d_Group file).
Regards

AP's picture

Dear Forum Supervisor,

Thanks for the feedback, i dont understand enough custom memory allocators(does this mean OCCT made an equivalent function to the malloc/realloc/free trio) in any case is there a way of telling it to not Hoard, or to set a limit on the Hoarding Bank(dont know how its called). I know this is branching off into a memory discussion, when we call the Standard::Purge is it releasing the memory back to the operating system or just clearing the memory and keeping it in reserve?

and is there another method besides purge to free all memory used by occ, like clearing the operating system clipboard ( i had to do this once in catia when using many copy paste operations).

i already know sometimes i am going to need a lot of memory like for example if i build say 10,000,000 doubly curved panels on a surface and i know it will consume roughly 6 gigs of memory once it gets close to the last panel to be built.

how can you tell the memory manager apriori take a 6 gigabyte chunk of memory and reserve it only for my instance of occ, instead of requesting memory on demand.

in the documentation you say:
MMGT_MMAP when set to 1 (default), large memory blocks are allocated using memory mapping functions of the operating system; if set to 0, they will be allocated in the C heap by malloc();

can you tell the memory manager allocate_large_memoryblock(6 x 10^9)

I havent done any Objective C, but i read they have an object called an NSAutoreleasePool which creates a context for temporary objects, sort of a mini heap or a stack that grows, then at the end of consuming the objects you delete the autoreleasepool and all objects created in it get deleted, would be awesome if OCC had a similar mechanism.

maybe defined as 2 preprocessor macros

@BeginTempMemoryPool(nameofpool,6gigs)
// any object created by handle between these two macros are allocated inside this temporary memory pool.
@EndTempMemoryPool // releases the memory pool.

///// about the self contained test //////

How should i structure this self contained test.
is there an example of one, online done in c++?

should it be a visual studio project?
is it literally a project file that you guys can just hit play,
for example it builds a view
and the interactive context
builds an array of randompoints in space
feeds them to the point_cloud ,
adds it to the interactive context ,
displays it,
and it closes?

Best,

AlexP

Forum supervisor's picture

Dear AlexP,
I would suggested you to overlook section "2.3. Memory Management in Open CASCADE" of the "Foundation Classes User's Guide". I hope it may help you to find answers to main part of your questions.
Concerning self-contained test.
There is no any special requirements for this except only natural one: the provided information should allow to the third party easy reproduce the reported problem. The problem should be localized to minimal piece of code (without extra details of your application) allowing to understand, run your code and reproduce the problem. For sure it should be supported by some environment details like version of OS, version of used compiler, debug/ or release mode and etc.

AP's picture

Dear Forum Supervisor.

Thank You for the guindance...

Best,

AlexP