I would like to know if mini batch is supported please.

Hello, I have an existing project that uses different types of graph neural networks. Other neural networks from the official dgl implementations are working normally, but the dgl version of your code experiences GPU memory leaks. Which version of dgl are you using?

I found for some tasks that I'm working on personally that the MPNN-style architecture does not perform well, no matter the aggregators or scalers that are used. Even the simple sum and mean aggregators perform less well than their GIN and GCN cousins.

For this reason, I propose to add the following simpler architecture as a variant of the PNA layer, which doesn't use the MPNN attention mechanism, but instead aggregates the neighbours in a similar way than CNN, GCN and GIN layers. An obvious drawback is the lack of edge features, but on my personal project on a molecular dataset, edge features seem to cause more overfit.

I propose to add it in the file pna/models/dgl/pna_layer.py. I did not implement it in pytorch-geometric or standard pytorch.

class PNASimpleLayer(nn.Module):

    def __init__(self, in_dim, out_dim, aggregators, scalers, avg_d, dropout, batch_norm, activation,
                posttrans_layers=1, residual=False):
        """
        A PNA layer that simply aggregates the neighbourhood (similar to GCN and GIN),
        without using the attention mechanism of the MPNN. It does not support edge features.

        :param in_dim:              size of the input per node
        :param out_dim:             size of the output per node
        :param aggregators:         set of aggregation function identifiers
        :param scalers:             set of scaling functions identifiers
        :param avg_d:               average degree of nodes in the training set, used by scalers to normalize
        :param dropout:             dropout used
        :param batch_norm:          whether to use batch normalisation
        :param posttrans_layers:    number of layers in the transformation after the aggregation
        """
        super().__init__()

        # retrieve the aggregators and scalers functions
        aggregators = [AGGREGATORS[aggr] for aggr in aggregators.split()]
        scalers = [SCALERS[scale] for scale in scalers.split()]

        self.aggregators = aggregators
        self.scalers = scalers
        self.in_dim = in_dim
        self.out_dim = out_dim
        self.dropout = dropout
        self.batch_norm = batch_norm

        self.batchnorm_h = nn.BatchNorm1d(out_dim)
        self.activation = activation
        self.posttrans = MLP(in_size=(len(aggregators) * len(scalers) + 1) * in_dim, hidden_size=out_dim,
                            out_size=out_dim, layers=posttrans_layers, 
                            mid_activation=activation, last_activation=activation,
                            dropout=dropout, mid_b_norm=batch_norm, last_b_norm=batch_norm)
        self.avg_d = avg_d


    def reduce_func(self, nodes):
        h = nodes.mailbox['m']
        D = h.shape[-2]
        h = torch.cat([aggregate(h) for aggregate in self.aggregators], dim=1)
        h = torch.cat([scale(h, D=D, avg_d=self.avg_d) for scale in self.scalers], dim=1)
        return {'h': h}


    def forward(self, g, h):
        g.ndata['h'] = h

        # aggregation
        g.update_all(fn.copy_u('h', 'm'), self.reduce_func)
        h = torch.cat([h, g.ndata['h']], dim=1)

        # posttransformation
        h = self.posttrans(h)

        return h


    def __repr__(self):
        return '{}(in_channels={}, out_channels={})'.format(self.__class__.__name__, self.in_dim, self.out_dim)

Hi, thank you very much for your sharing of your code. Is there any way that in which kind of code, you can provide a examples how to create the input and how to use the layer?

Hi @lukecavabarrett @gcorso

Thank you for this great work! I have a minor question here. Is it possible to use learnable scalers that could be adapted according to different inputs, instead of using degree-based scalers? Thank you!

Best,
Yongcheng

The type of device is either torch.device("cuda") or torch.device("cpu"). Thus, device == 'cuda' will always be False and torch.cuda.manual_seed(params['seed']) will not work.

It seems that args.divide_input_first and args.divide_input_first will always be True, even if we set --divide_input_first=False in the command line. A similar question can be found here.

Hi all! First of all, great paper, I could definitely see it boosting GNN research.

I believe there is an issue with the default parameters in models/pna/train.py, line 11, namely:

parser.add_argument('--scalers', type=str, default='identity exp log', help='Scalers to use')

exp and log are not part of the scalers. Based on the paper (and the README) I assume you mean amplification and attenuation. Currently, as the code is, not providing scalers argument results in a KeyError.

Thanks!

Hi,
Could you tell me where is main_PCBA file?
Thank you,
Zhaoning

I think the calculation of average degrees contains an error when example.py is called.

The deg variable (defined in pna/models/pytorch_geometric/example.py) contains, at index i, the number of vertices in the dataset with degree i. This is passed to the init function of PNAConvSimple (in pna/models/pytorch_geometric/pna.py) to calculate the values for self.avg_deg.

But we end up calculating the average of the degree counts, not the average of the degrees. The following two (non-optimised) solutions would work:

• Edit example.py so deg is actually a list of degrees

deg = []
for data in dataset[split_idx['train']]:
    d = degree(data.edge_index[1], num_nodes=data.num_nodes, dtype=torch.long)
    deg += list(d)
deg = torch.tensor(deg, dtype=torch.long)

• Edit pna.py to handle binned frequency tensors (such as is returned by torch.bincount)

total_no_vertices = deg.sum()
bin_degrees = torch.arange(len(deg))
self.avg_deg: Dict[str, float] = {
    'lin': ((bin_degrees*deg).sum()/total_no_vertices).item(),
    'log': (((bin_degrees+1).log()*deg).sum()/total_no_vertices).item(),
    'exp': ((bin_degrees.exp()*deg).sum()/total_no_vertices).item(),
}

Apologies if I've misunderstood what the avg_deg variable is supposed to contain. Note I haven't checked elsewhere to see if this same issue arises in other files.

File "/Users/dimmy/miniconda3/lib/python3.8/site-packages/dgl/graph_index.py", line 53, in setstate
num_nodes, readonly, src, dst = state
ValueError: too many values to unpack (expected 4)