Neural Predicate Invention

The core function is the unsupervised or weakly-supervised generation of new symbolic predicates. The system analyzes data patterns and proposes new logical concepts (e.g., is_metastable(state) in chemistry or upward_trend(metric, window) in finance) that were not present in the initial knowledge base. This moves beyond predefined ontologies.

• Process: Often involves searching a space of possible predicate definitions.
• Goal: To find concepts that compactly explain observations or improve task performance.

Invented predicates are represented in a form compatible with gradient-based learning. Unlike static symbols in classic AI, these predicates are often embedded as continuous vectors or implemented as neural modules. This allows the system to refine the meaning and utility of a new predicate through backpropagation based on task loss.

• Key Benefit: The invented concept's definition can be softly adjusted during training for optimal integration with other neural components.

Newly invented predicates are immediately operational within a symbolic reasoning framework. They can be used as atoms in first-order logic rules, incorporated into a knowledge graph, or serve as conditions in a production system. For example, an invented predicate risk_factor(patient, X) can be used in a learned rule: IF risk_factor(patient, high) AND age(patient, >65) THEN recommend(test, Y).

• This creates a closed loop where reasoning guides invention and new inventions enhance reasoning.

Invention is not arbitrary; it's guided by objective functions related to explanatory power and task utility. Common drivers include:

• Minimum Description Length (MDL): Prefers predicates that allow the overall system (data + theory) to be described more succinctly.
• Improvement in Predictive Accuracy: A new predicate is retained if it improves the model's performance on a target task (e.g., classification, planning success).
• Increase in Inferential Efficiency: The predicate speeds up logical deduction or query answering.

This process often acts as the critical link between sub-symbolic perception (e.g., raw images, sensor data, text) and high-level reasoning. A neural network perceives the world and invents symbolic abstractions to describe it. These abstractions are then manipulated by a logical reasoner to make decisions or draw conclusions.

• Example: A vision system watching blocks-world videos might invent the predicate supports(object_a, object_b) to logically reason about stability, going beyond raw pixel data.

It provides a mechanism for autonomous knowledge acquisition. Starting with a seed knowledge base, the system can propose new facts and relations using invented predicates, thereby extending its own symbolic knowledge. This is a step towards lifelong learning in neuro-symbolic systems, where the agent's understanding of the world becomes richer and more structured over time without manual engineering.

• Contrasts with static knowledge bases that require human curation.

A foundational machine learning framework for neural predicate invention. ∂ILP learns logic programs (sets of Horn clauses) from examples using gradient-based optimization. It bridges symbolic rule induction with neural network training by representing logical predicates as differentiable functions.

• Core Mechanism: Treats the truth values of logical atoms as continuous probabilities, enabling gradient flow through logical deductions.
• Role in Invention: The system can propose new predicate symbols as part of the learned program to better explain the training data, which is the essence of predicate invention.
• Example: Given examples of family relationships, a ∂ILP system might invent a predicate for 'grandparent' by composing learned 'parent' rules.

A neuro-symbolic approach that extends traditional logic programming (e.g., Prolog) by representing predicates and rules as learnable, differentiable neural modules. This creates a substrate where predicate invention can occur.

• Architecture: Facts and rules are embedded in a continuous vector space. Unification and proof search become differentiable operations.
• Invention Context: New neural predicates can emerge as specialized sub-networks or clusters within the embedding space that capture recurring, useful patterns not present in the initial symbolic vocabulary.
• Contrast with ∂ILP: Often focuses more on scaling probabilistic reasoning with neural networks rather than strictly learning discrete logical programs from scratch.

The technique of extracting interpretable symbolic knowledge—such as rules, concepts, or predicates—from a trained neural network. It is a downstream application that can utilize invented predicates.

• Process: A 'teacher' neural network (which may have learned implicit concepts) is analyzed to produce a 'student' symbolic model.
• Connection to Invention: The distilled symbols or rules may represent predicates that were not explicitly defined in the original training data but were discovered by the neural network during learning. Neural predicate invention can be seen as a form of continuous or in-training distillation.
• Use Case: Explaining the decision logic of a complex deep learning model by distilling it into a set of invented, human-readable predicates and rules.

An architecture that applies graph neural networks (GNNs) to structured, symbolic knowledge representations like knowledge graphs. It enables relational reasoning and is a common substrate for learning and inventing relational predicates.

• Mechanism: Entities and relation types are embedded. GNNs perform message-passing to infer new links or classify nodes.
• Predicate Invention in Graphs: The model can learn to represent novel composite relations (invented predicates) as paths or subgraph patterns in the embedding space. For example, it might invent a 'potential-collaborator' predicate by combining co-authorship and research-topic similarity links.
• Application: Directly used for tasks like neural knowledge base completion, where inventing new relation types can improve link prediction.

AI systems that perform inference to the best explanation for observed phenomena. Abduction is a core logical mode that drives the need for predicate invention.

• Logical Form: Given an observation O and a background theory T, find a hypothesis H (often involving new entities or predicates) such that T + H logically entails O.
• Role of Invention: When the background theory is insufficient to explain observations, the system must invent new hypothetical predicates (H) to form a coherent explanation. A neural abduction engine would use neural networks to generate and score these invented hypotheses.
• Example: In medical diagnosis, observing a complex of symptoms (O) might lead to the abduction (and thus invention) of a new syndrome predicate (H) that explains them all.

A framework that reformulates logical operations (AND, OR, NOT, implication) into continuous, differentiable functions. This is the essential mathematical glue that makes gradient-based neural predicate invention possible.

• Core Idea: Replace binary truth values with continuous values in [0,1] and use fuzzy logic or product real logic operators that have well-defined gradients.
• Enabler for Invention: By making logic differentiable, the definitions and applications of predicates become parameters that can be optimized via gradient descent. A system can smoothly adjust what a novel predicate means to better satisfy logical constraints and data.
• Frameworks: Logic Tensor Networks (LTNs) are a prominent implementation, using fuzzy logic to inject first-order logical knowledge into deep learning models.