3D Point Cloud and Manifold datasets in OGB

97 views
Skip to first unread message

ckjo...@gmail.com

unread,
Aug 31, 2020, 3:50:25 AM8/31/20
to Open Graph Benchmark
Hi OGB Team,

I was wondering if there are plans to incorporate 3D Point Cloud and Manifold tasks such as shape recognition or segmentation into the benchmarks?

Here are some SotA references using GNNs:

I am not very familiar with this line of work, so I would also be happy to hear if someone from the community has any suggestions of good datasets for benchmarking GNNs in the 3D Computer Vision domain. 

Best,
Chaitanya

Open Graph Benchmark

unread,
Aug 31, 2020, 3:56:35 PM8/31/20
to Open Graph Benchmark
Hi Chaitanya,

Great suggestion. We do not have an immediate plan to include 3D Point Cloud benchmark, but we are definitely open to concrete suggestion from the community.

Best,
OGB Team

Chaitanya K. Joshi

unread,
Aug 31, 2020, 9:40:56 PM8/31/20
to Open Graph Benchmark
Hopefully, someone more experienced in this line of work can continue this thread with some suggestions! I am interested in point clouds as they seem a good benchmark for leveraging and studying GNN depth. 

More resources:

Weihua Hu

unread,
Aug 31, 2020, 11:50:39 PM8/31/20
to Open Graph Benchmark
As far as I know, both ShapeNet and ModelNet are already available via Pytorch Geometric, and are easily accessible. We would like to include non-original datasets in OGB only when we can provide a significant improvement for the community, e.g., standardization effort, more meaningful splits, better graph representation, etc. Is this the case for ShapeNet and ModelNet?Also, if people feel existing 3D benchmarks are problematic, and would like to make effort to create an original one, please let us know. We are happy to help. More generally, we are looking forward to hearing from the community any problems you have felt about the datasets and concrete suggestions you have to resolve the problems.Best,
Weihua

matthi...@tu-dortmund.de

unread,
Sep 1, 2020, 1:19:59 AM9/1/20
to Open Graph Benchmark
Hey everyone,

this is an interesting discussion, especially because I feel that there are "real-world" point cloud classification datasets missing for point clouds. For both ShapeNet and ModelNet, point clouds are derived by sampling points from mesh surfaces, which doesn't seem very realistic to me. If you do know of any, please let us know :)

Although GNNs are at the top of performance on point cloud networks, I'm not sure how useful it would be to include any in OGB:
* GNNs for point clouds can not typically be applied to other domains
* It's hard to standardize the graph representation, and it's usually much better to just provide the raw points and let the user decide which graph representation works best for his/her use-case

Best,
Matthias

Chaitanya K. Joshi

unread,
Sep 1, 2020, 2:51:46 AM9/1/20
to Open Graph Benchmark
Thanks for chiming in! I am really no expert, but just to continue the discussion:

M: For both ShapeNet and ModelNet, point clouds are derived by sampling points from mesh surfaces, which doesn't seem very realistic to me.
C: From my understanding, sampling from 2D manifolds (i.e. mesh surfaces?) is a popular approach today because of scalability. For e.g., this recent CVPR'20 paper talks about this: 
> " In computer vision and graphics, early attempts at applying deep learning to 3D shapes were based on dense voxel representations or multiple planar views. These methods suffer from three main drawbacks, stemming from their extrinsic nature: high computational cost of 3D convolutional filters, lack of invariance to rigid motions or non-rigid deformations, and loss of detail due to rasterisation. A more efficient way of representing 3D shapes is modeling them as surfaces (two-dimensional manifolds). In computer graphics and geometry processing, a popular type of efficient and accurate discretisation of surfaces are meshes or simplicial complexes, which can be considered as graphs with additional structure (faces)."

M: * GNNs for point clouds can not typically be applied to other domains
* It's hard to standardize the graph representation, and it's usually much better to just provide the raw points and let the user decide which graph representation works best for his/her use-case
C: Indeed, SotA approaches for point clouds seem to consider the input as a set of point and use the k-nearest neighbors of each point to dynamically compute the graph at each GNN layer. However, I feel that the layer update equation/GNN equations are usually general enough for them to be applied to fixed graphs from other domains, too. I am thinking of this or this one. 

Guohao Li

unread,
Sep 4, 2020, 6:50:02 AM9/4/20
to Open Graph Benchmark
Hi everyone,

Thanks Chaitanya for bringing up this interesting discussion. I agree that benckmarking 3D classification/segmentation is very important. As far as I know, there are several recent works aim to solve this problem:

*For classfication, ModelNet was proposed in 2015. After 5 years, people still stick to evaluate their classification methods on ModelNet due to the lack of standardized larger dataset. ScanObjectNN (ICCV2019 oral) is one recent attempt to alleviate this pain. They proposed a 3D object dataset obtained from real-world 3D scans and benchmarked serval popular 3D methods including 3DmFV, PointNet, SpiderCNN, PointNet++, DGCNN, PointCNN. Interestingly, DGCNN as a graph NN method achieves very decent performance.

*For semantic segmentation,  torch-points3d benmarks point network methods on popular segmentation datasets such as ScannetS3DISShapenet. PartNet are also working on an online benchmark like Scannet. However, graph-based methods are missing.

*Other dataset, Alibaba releases two large 3D indoor scence/object dataset this year: 3D-FRONT and 3D-FUTURE.

In short, the works above are not designed for graph learning / GNN methods specifically. I think it makes sense to have pointcloud /mesh datasets in OGB. It would be very helpful for uncovering the power of GNNs method in 3D and also provides a broader graph domain for the graph/geometric learning community to play with.

Best,
Guohao

Chaitanya K. Joshi

unread,
Sep 8, 2020, 4:03:00 AM9/8/20
to Open Graph Benchmark
Thanks for the resources, Guohao! Excited to make my way through them. 

I also recently encountered a talk by Prof. M. Bronstein, where he also dives into this question of what are the best input representations for 3D shapes: https://www.youtube.com/watch?v=PLGcx65MhCc (forward to the 37:15 mark). It may be of interest to participants here.
Reply all
Reply to author
Forward
0 new messages