Edge weights and constraints¶

This notebook explains the optional edge-level metadata columns supported by edge2torch edgelists:

• initial_weight
• constraint

The required edgelist columns are still source and target. These optional columns do not change how nodes, inputs, outputs, or graph connectivity are inferred. They only control how selected edge weights are initialized or constrained in the compiled PyTorch model.

Optional columns¶

initial_weight defines the initial effective weight for an edge.

constraint defines how the edge weight behaves during training. Supported values are:

Value	Meaning
unconstrained	Trainable edge weight; may become positive or negative.
positive	Trainable edge weight constrained to remain positive.
negative	Trainable edge weight constrained to remain negative.
fixed	Non-trainable edge weight fixed to initial_weight.

For constrained trainable edges, edge2torch learns an unconstrained latent parameter and transforms it into the effective edge weight during the forward pass. Positive and negative constraints are implemented with the softplus function:

$$ \operatorname{softplus}(\theta) = \log(1 + e^\theta) $$

The softplus transform is a smooth positive alternative to ReLU: it is always greater than zero, but remains differentiable around zero.

For a positive constraint, the effective edge weight is

$$ w = \operatorname{softplus}(\theta) $$

For a negative constraint, the effective edge weight is

$$ w = -\operatorname{softplus}(\theta) $$

where $\theta$ is the trainable latent parameter and $w$ is the effective edge weight used by the compiled model.

Both columns are sparse and row-wise optional:

• missing initial_weight means default PyTorch-style initialization for that edge
• missing constraint means unconstrained
• constraint="fixed" requires an initial_weight in the same row

The columns can therefore be used independently. For example, one edge can have a custom initial weight, another edge can have only a sign constraint, and a third edge can use the default behavior.

Imports¶

In [1]:

import numpy as np
import pandas as pd
import torch
from IPython.display import display

import edge2torch as e2t

import numpy as np import pandas as pd import torch from IPython.display import display import edge2torch as e2t

Define an edgelist with sparse edge metadata¶

The following graph has four edges. Each edge demonstrates a different metadata case:

1. a positive-constrained edge initialized to 0.25
2. a negative-constrained edge initialized to -0.50
3. a fixed edge with value 0.75
4. an unconstrained edge with default initialization

In [2]:

edgelist = pd.DataFrame(
    {
        "source": [
            "feature_a",
            "feature_b",
            "hidden_pos",
            "hidden_neg",
        ],
        "target": [
            "hidden_pos",
            "hidden_neg",
            "prediction",
            "prediction",
        ],
        "initial_weight": [
            0.25,
            -0.50,
            0.75,
            np.nan,
        ],
        "constraint": [
            "positive",
            "negative",
            "fixed",
            None,
        ],
    }
)

display(edgelist)

edgelist = pd.DataFrame( { "source": [ "feature_a", "feature_b", "hidden_pos", "hidden_neg", ], "target": [ "hidden_pos", "hidden_neg", "prediction", "prediction", ], "initial_weight": [ 0.25, -0.50, 0.75, np.nan, ], "constraint": [ "positive", "negative", "fixed", None, ], } ) display(edgelist)

	source	target	initial_weight	constraint
0	feature_a	hidden_pos	0.25	positive
1	feature_b	hidden_neg	-0.50	negative
2	hidden_pos	prediction	0.75	fixed
3	hidden_neg	prediction	NaN	None

Compile the graph¶

The metadata columns are consumed by compile_graph() automatically when they are present. No extra API arguments are needed.

In [3]:

model, artifact = e2t.compile_graph(
    edgelist=edgelist,
    backend="recurrent",
    quiet=True,
)

type(model.recurrent).__name__

model, artifact = e2t.compile_graph( edgelist=edgelist, backend="recurrent", quiet=True, ) type(model.recurrent).__name__

Out[3]:

'ConstrainedMaskedLinear'

The normalized graph stored in the artifact preserves the metadata. Missing constraints are normalized to unconstrained; missing initial weights remain missing because they mean “use default initialization.”

In [4]:

display(artifact.graph.edges)

display(artifact.graph.edges)

	source	target	initial_weight	constraint
0	feature_a	hidden_pos	0.25	positive
1	feature_b	hidden_neg	-0.50	negative
2	hidden_pos	prediction	0.75	fixed
3	hidden_neg	prediction	NaN	unconstrained

Inspect the effective recurrent weight matrix¶

The constrained layer stores trainable latent parameters internally and computes the effective weight matrix during the forward pass.

In [5]:

def recurrent_weight_table(model):
    weight = model.recurrent.effective_weight.detach().cpu()
    return pd.DataFrame(
        weight.numpy(),
        index=model.node_names,
        columns=model.node_names,
    )


display(recurrent_weight_table(model))

def recurrent_weight_table(model): weight = model.recurrent.effective_weight.detach().cpu() return pd.DataFrame( weight.numpy(), index=model.node_names, columns=model.node_names, ) display(recurrent_weight_table(model))

	feature_a	feature_b	hidden_neg	hidden_pos	prediction
feature_a	-0.403541	0.220338	0.415684	-0.061539	0.388678
feature_b	-0.281428	-0.274804	-0.386649	0.413306	-0.371817
hidden_neg	-0.343529	-0.500000	-0.117810	0.412458	-0.342862
hidden_pos	0.250000	-0.119451	-0.310758	0.171553	0.304306
prediction	0.403330	0.020690	0.293822	0.750000	0.350958

In the weight table, rows are target nodes and columns are source nodes. In other words, the value at row target and column source corresponds to the edge

source -> target

The explicitly initialized edges have the requested effective values:

• feature_a -> hidden_pos starts at 0.25
• feature_b -> hidden_neg starts at -0.50
• hidden_pos -> prediction is fixed at 0.75

The edge hidden_neg -> prediction has no initial_weight, so it uses the default initialization.

In [6]:

weights = recurrent_weight_table(model)

selected_edges = pd.DataFrame(
    {
        "edge": [
            "feature_a -> hidden_pos",
            "feature_b -> hidden_neg",
            "hidden_pos -> prediction",
            "hidden_neg -> prediction",
        ],
        "effective_weight": [
            weights.loc["hidden_pos", "feature_a"],
            weights.loc["hidden_neg", "feature_b"],
            weights.loc["prediction", "hidden_pos"],
            weights.loc["prediction", "hidden_neg"],
        ],
    }
)

display(selected_edges)

weights = recurrent_weight_table(model) selected_edges = pd.DataFrame( { "edge": [ "feature_a -> hidden_pos", "feature_b -> hidden_neg", "hidden_pos -> prediction", "hidden_neg -> prediction", ], "effective_weight": [ weights.loc["hidden_pos", "feature_a"], weights.loc["hidden_neg", "feature_b"], weights.loc["prediction", "hidden_pos"], weights.loc["prediction", "hidden_neg"], ], } ) display(selected_edges)

	edge	effective_weight
0	feature_a -> hidden_pos	0.250000
1	feature_b -> hidden_neg	-0.500000
2	hidden_pos -> prediction	0.750000
3	hidden_neg -> prediction	0.293822

Constraints during optimization¶

Positive and negative constraints are enforced by parameterization. The optimizer updates latent parameters, while the effective edge weights are computed as constrained values during the forward pass.

For example:

• positive edges use a positive transform of a trainable latent parameter
• negative edges use the negative of that transform
• fixed edges are stored as non-trainable buffers

The following small optimizer step demonstrates that the constrained and fixed edges keep their intended behavior.

In [7]:

x = torch.tensor(
    [
        [1.0, 2.0],
        [0.5, 1.5],
    ]
)

fixed_before = weights.loc["prediction", "hidden_pos"]

optimizer = torch.optim.SGD(model.parameters(), lr=0.1)

for _ in range(5):
    optimizer.zero_grad()
    output = model(x)
    loss = output.pow(2).sum()
    loss.backward()
    optimizer.step()

weights_after = recurrent_weight_table(model)

checks = pd.DataFrame(
    {
        "edge": [
            "feature_a -> hidden_pos",
            "feature_b -> hidden_neg",
            "hidden_pos -> prediction",
        ],
        "condition": [
            "positive after optimization",
            "negative after optimization",
            "fixed value preserved",
        ],
        "value": [
            weights_after.loc["hidden_pos", "feature_a"],
            weights_after.loc["hidden_neg", "feature_b"],
            weights_after.loc["prediction", "hidden_pos"],
        ],
        "passes": [
            weights_after.loc["hidden_pos", "feature_a"] > 0,
            weights_after.loc["hidden_neg", "feature_b"] < 0,
            np.isclose(
                weights_after.loc["prediction", "hidden_pos"],
                fixed_before,
            ),
        ],
    }
)

display(checks)

x = torch.tensor( [ [1.0, 2.0], [0.5, 1.5], ] ) fixed_before = weights.loc["prediction", "hidden_pos"] optimizer = torch.optim.SGD(model.parameters(), lr=0.1) for _ in range(5): optimizer.zero_grad() output = model(x) loss = output.pow(2).sum() loss.backward() optimizer.step() weights_after = recurrent_weight_table(model) checks = pd.DataFrame( { "edge": [ "feature_a -> hidden_pos", "feature_b -> hidden_neg", "hidden_pos -> prediction", ], "condition": [ "positive after optimization", "negative after optimization", "fixed value preserved", ], "value": [ weights_after.loc["hidden_pos", "feature_a"], weights_after.loc["hidden_neg", "feature_b"], weights_after.loc["prediction", "hidden_pos"], ], "passes": [ weights_after.loc["hidden_pos", "feature_a"] > 0, weights_after.loc["hidden_neg", "feature_b"] < 0, np.isclose( weights_after.loc["prediction", "hidden_pos"], fixed_before, ), ], } ) display(checks)

	edge	condition	value	passes
0	feature_a -> hidden_pos	positive after optimization	0.253219	True
1	feature_b -> hidden_neg	negative after optimization	-0.490694	True
2	hidden_pos -> prediction	fixed value preserved	0.750000	True

Reset behavior¶

Calling reset_parameters() restores explicit edgelist-defined initial weights. Edges without explicit initial_weight are reinitialized using the default initializer.

In [8]:

model.recurrent.reset_parameters()

display(recurrent_weight_table(model))

model.recurrent.reset_parameters() display(recurrent_weight_table(model))

	feature_a	feature_b	hidden_neg	hidden_pos	prediction
feature_a	0.108298	0.200703	-0.441670	-0.094280	0.279052
feature_b	-0.395788	0.378382	-0.192658	-0.101646	-0.215980
hidden_neg	-0.322694	-0.500000	-0.337053	-0.049620	0.430820
hidden_pos	0.250000	-0.229037	0.225490	0.147331	0.168374
prediction	0.381832	-0.136737	-0.245201	0.750000	0.329351

Feedforward skip edges¶

The feedforward backend may internally expand skip edges through pseudo nodes. The edge metadata still belongs to the original logical edge.

For a skipped edge, edge2torch assigns the original edge metadata to the final internal edge into the original target node. Internal pseudo-routing edges remain default-initialized and unconstrained because pseudo-node activations are overwritten as pass-through copies.

In [9]:

skip_edgelist = pd.DataFrame(
    {
        "source": [
            "feature_a",
            "feature_a",
            "hidden",
            "middle",
        ],
        "target": [
            "hidden",
            "prediction",
            "middle",
            "prediction",
        ],
        "initial_weight": [
            0.20,
            0.75,
            0.30,
            -0.40,
        ],
        "constraint": [
            "positive",
            "fixed",
            "positive",
            "negative",
        ],
    }
)

feedforward_model, feedforward_artifact = e2t.compile_graph(
    edgelist=skip_edgelist,
    backend="feedforward",
    quiet=True,
)

display(feedforward_artifact.execution_plan.expanded_edges)

skip_edgelist = pd.DataFrame( { "source": [ "feature_a", "feature_a", "hidden", "middle", ], "target": [ "hidden", "prediction", "middle", "prediction", ], "initial_weight": [ 0.20, 0.75, 0.30, -0.40, ], "constraint": [ "positive", "fixed", "positive", "negative", ], } ) feedforward_model, feedforward_artifact = e2t.compile_graph( edgelist=skip_edgelist, backend="feedforward", quiet=True, ) display(feedforward_artifact.execution_plan.expanded_edges)

	source	target	initial_weight	constraint
0	feature_a	hidden	0.20	positive
1	feature_a	__edge2torch_pseudo__feature_a__prediction__la...	NaN	unconstrained
2	__edge2torch_pseudo__feature_a__prediction__la...	__edge2torch_pseudo__feature_a__prediction__la...	NaN	unconstrained
3	__edge2torch_pseudo__feature_a__prediction__la...	prediction	0.75	fixed
4	hidden	middle	0.30	positive
5	middle	prediction	-0.40	negative

In the expanded execution plan, the internal pseudo-routing edges for the skip path have missing initial_weight and unconstrained constraints. The final pseudo-to-target edge carries the original initial_weight=0.75 and constraint="fixed" from the user-provided skip edge.

Practical guidance¶

Use initial_weight when a prior value is meaningful or when reproducible edge-specific initialization matters.

Use constraint when the allowed domain of an edge weight is part of the model definition. Typical examples include nonnegative effects, nonpositive effects, or fixed known coefficients.

Do not use positive or negative if the sign is only a weak guess. In that case, either leave the edge unconstrained or use the value only as an initial_weight.

Edge constraints apply to connection weights before any activation function. If you interpret positive and negative constraints as directional effects, that interpretation is clearest when downstream activations are monotonic.