Member Avatar for G-Do

Hi guys,

I'm trying to learn the principles of artificial intelligence. I just finished coding up a belief net and a corresponding EM algorithm that learns the CPTs for latent nodes using training data (if you're unfamiliar with the lingo, you're probably going :eek: right now). While the program (written in Python, of course) works and is pretty quick, it is also pretty unstructured - I had difficulty figuring out how to best represent probability computations, so parts of the algorithm look kind of awkward.

Are there any good, complete AI/machine learning packages in Python - are there any which incorporate Bayesian belief nets? Any which represent the belief net, the update procedure, and EM algorithms to find the parameters at latent nodes? How about BioPython? Since I'm a computational biology guy, that would be right up my alley, but I'm not sure how complete the BioPython package is. How about Orange - how easily does it lend itself to bioinformatics-type applications? Does anyone here have any experience with this stuff? If not, I guess I'll start plumbing it myself, but I'd appreciate it if anyone could save me some time :cheesy:.

Here's a link to BioPython's homepage, if anyone is interested in learning:

BioPython

And here is Orange:

Thanks!

  • 5 Contributors
  • 5 Replies
  • 358 Views
  • 5 Years Discussion Span
  • Latest Post Latest Post by Gael Varoquaux

Dani AI

Generated

Good call — implementing a belief network plus EM yourself is the best way to learn the plumbing, but these days several Python libraries already handle representation, inference and EM so you can focus on model design and data issues. was right that scikit-learn is excellent for many ML algorithms, but for explicit Bayesian networks you’ll want a BN-focused package.

Consider these libraries that directly support Bayesian networks and parameter learning (including EM): pgmpy (pure-Python, has an ExpectationMaximization estimator), pomegranate (C/Cython-backed, fast BN fitting and imputation), pyagrum (C++ core with a Python wrapper and extensive BN learning tools), and the Python bnlearn package (easy pipelines for structure and parameter learning). (pgmpy.org)

If you want a full Bayesian treatment (posteriors for latent nodes rather than point estimates), look at probabilistic programming tools such as PyMC, Pyro or TensorFlow Probability — they give MCMC/VI engines for uncertainty quantification but require a different workflow than EM. EM is fast and gives ML estimates; PPLs give full posterior uncertainty and are better when credible intervals or hierarchical priors matter. (pymc.io)

Practical starter: prototype structure+EM with pgmpy, use pomegranate/pyagrum for speed if datasets are large, and move to PyMC/Pyro/TFP when you need full Bayesian inference. Quick example (pgmpy-style):

from pgmpy.models import DiscreteBayesianNetwork
from pgmpy.estimators import ExpectationMaximization
import pandas as pd

data = pd.read_csv('observed.csv')   # missing values as NaN
model = DiscreteBayesianNetwork([('A','B'), ('C','B')], latents={'B'})
em = ExpectationMaximization(model, data)
cpds = em.get_parameters(latent_card={'B':3})
model.add_cpds(*cpds)

Notes: encode categories consistently, impute/represent missing as NaN, run multiple random restarts (EM finds local optima), validate with held-out data or BIC/AIC, and use BioPython for sequence I/O / scikit-learn for preprocessing where helpful. (pgmpy.org)

Recommended Answers

All 5 Replies

Member Avatar for bumsfeld

Thanks G-do for informing us over the interesting medical work you are doing! Orange sounds like Python based data mining module, things that should be explored. Slovenia is just like Austria, very nice with much history and great people.

Member Avatar for Ene Uran

I took a look at BioPython, very nice work they have done!

Member Avatar for G-Do

BioPython does simplify things a good deal, doesn't it? :cheesy: If I get a free moment, I might do a BioPython/Orange code snippet with some basic biology background, the goal of which would be to describe how to code up a simple research program.

Member Avatar for vegaseat

I will be waiting for that code snippet! Sounds all very interesting to me too!

Member Avatar for Gael Varoquaux

Have a look at the scikit-learn. While it does not have any code for belief nets, it has many state of the art algorithms that can be used for machine learning with Python.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.