The Future of
LearnLib

RALib reintegration

Provide the features of RALib natively in AutomataLib and LearnLib

Register automata in AutomataLib

• done:
  • separate RAs and RMMs
  • re-write jStateExplorer for equivalence testing
• in progress:
  • switch from jConstraints to konstraints
• todo:
  • serialization

Learning RA/RMM in LearnLib

• in progress:
  • basic restructuring of oracles / theories / learners
  • RA learning
• todo:
  • native RMM learning

A unified benchmarking platform

We want to provide a benchmarking platform that allows users to easily compare their own learner/tester with existing implementations

Technical Aspects

• use docker as runtime environment
  • use CI/CD to support multi-platform manifests (AMD64, ARM64, etc.)
  • provide an out-of-the-box experience for multiple environments
  • persistence through container lifetime
  • optional inputs through docker volumes/mounts
• expose services via HTTP
  • support multiple programming languages/operating systems directly
  • automatically generate OpenAPI documentation for endpoints

Use cases

1: you want to benchmark your own learner

• set up the learning process
  • select the SUL
  • select the EQO(s)
  • returns alphabet definition and experiment ID
• run the learning process
  • repeatedly pose (batches of) queries to an endpoint
    • receive responses
  • submit tentative hypothesis to an endpoint
    • receive counterexample
• extract results
  • export statistics

Use cases

2: you want to benchmark your own tester

• set up the learning process
  • select the SUL
  • select the learner
  • returns alphabet definition and experiment ID
• (busy-) wait for a tentative hypothesis
  • read the hypothesis
  • submit test queries
  • returns whether a counterexample has been found or more test queries are required
• extract results
  • export statistics

Use cases

3: you want to benchmark existing configurations

• set up the learning process
  • select the SUL
  • select the EQO(s)
  • select the learner(s)
  • returns (multiple) experiment ID(s)
• extract results
  • export statistics

Challenges

• client-server architecture
  • client needs to take the initiative vs. client needs to provide an endpoint
  • how to implement deeply stateful communication (e.g., adaptive queries)
• alphabet symbol abstractions
  • how to deal with complex inputs (MMLTs, RAs, etc.)
• interface
  • do we need a graphical UI?

AI integration

What does the rise of AI mean for automata learning?

learning agents

• Direction 1:
  • the learner remains the initiator
  • use AI to answer membership queries
  • use AI to answer equivalence queries
• Direction 2:
  • offer automata learning as a service
  • AI may autonomously contact a LearnLib agent
  • provide feedback via counterexamples

Discussion

Feedback? Opinions? Ideas?