Archive for AI

Pyipopt, Python marries with Ipopt

Ipopt is a leading nonlinear optimization tool. There are a bunch of interfaces, e.g. C/C++ standard interfaces, Java and MATLAB interface, FORTRAN interface for programmers.  However, no existing interface to python was presented. I started to connect ipopt with python two months ago inspired by OpenOpt. After lots of extensive development and nasty testing,  today, I proudly announce that the python interface of ipopt, aka, pyipopt, version alpha, is ready to go.

With the help of this module, in python, you can just use

import pyipopt
nlp = pyipopt.create(n, xl, xu, m, gl, gu, nnzj, nnzh, f, gradf, g, gradg, h, apply_new)
solution = nlp.solve(x0)

to get your NLP problem which n variables (xl, xu are boundaries for variables), m constraints (gl, gu are boundaries for constraints), and objective function f (gradf is the gradient, h is the Hessian matrix) with constraints g (gradg is the Jacobian value) optimized by IPOPT.

The package is available via google code: http://code.google.com/p/pyipopt/   An OpenOpt hooker is under development.

The code has BSD license. you can use this module whatever you want. For bug report and other related things, reach me at youxu AT wustl.edu.

To write a  package is not rocket science, but it might save you some time in your research.

[Special thanks to Dmitrey Kroshko from OpenOpt and Scipy community; Dominique Orban from Nlpy community]

Comments (1)

请懂量子力学的一起帮我参详一篇论文

文章名:

Ultimate Physical Limits to Computation [计算的终极物理极限]

作者: MIT 的 Seth Lloyd,  量子计算领域顶尖科学家.

概要:  计算机是一种物理系统,因此肯定服从物理系统的规律。文章研究了一千克的计算机最多一秒能执行多少次逻辑运算,最大存储容量是多少,并且给出了定量的结果。所以, 摩尔定律不能永续。

这些问题和我以前提到的比特每焦耳问题是很相似,但是我物理基础有限,不能全部理解。
本文发在 2000 年 Nature.地址: 

http://puhep1.princeton.edu/~mcdonald/examples/QM/lloyd_nature_406_1047_00.pdf

有兴趣的直接给我发邮件或者gtalk: xu.mathena [A@T] gmail.com

Comments (3)

语言模型与语言识别程序 -1

[先说一下, 上次的语言识别程序的思路来自 Python Cookbook, 我只是略作修改, 并没有什么太大的贡献, 这个系列可能包含三到四篇文章. 底下我可能会用朴素贝页斯分类器再写一个然后比较两者效果]

以前我在翻译怎样写一个拼写检查器中, 就提到了(概率)语言模型这个东西. 什么是语言模型呢, 简单的说, 就是这个语言当中, 每个词出现的概率分布是有一定规律的, 与其他因素无关的. 如果我们要写一个汉语到英文的翻译器, 那么, 理论上说, 我们翻译器输出来的结果当中每个单词出现的概率, 应该和英文当中每个词出现的概率一样. 试想一下, the 这个单词在英文中常常出现, 而如果我们的翻译器对于任意汉语的翻译结果中, 故意调低或者错误设置 the 出现的概率, 造成翻译出来的结果几乎不出现 the, 或者出现的概率出奇的奇怪, 那么, 这段翻译出来的话, 它怎么看也不会像英语. 同样的道理, 一个拼写检查器可以把任意拼错的单词纠正过来, 在纠正过程中必须照顾到语言模型. 如果 teh 可以被改正为 the 和 tek, 那么我们应该优先考虑 the 才对, 因为 the 被使用的概率比 tek 大. (统计语言模型整体是很大的一块, 大家可以去读 Google 黑板报数学之美系列得到一个大致的感觉. 我这里只用到语言模型, 用不到条件概率)

基于这个假设, 我们自然的得到一个主意: 好呀, 假如两个文本的语言模型类似, 我们就认为他们是同种语言写成的. 对的, 怎么衡量这个类似呢, 我们可以用数学的方法: 统计任何两个语言模型之间的”距离”, 然后选取最小的. 这个地方距离随便怎么定义, 可以用向量夹角, 可以用欧式距离. 但是, 现实的问题不是这么简单. 首先, 如果每个词都对应一个词频的话, 得到的向量是高维的. 除非有一个包含各种语言单词的全集, 否则很难说这个向量一共要有多少个维度. 而且高维的随机向量还往往以很大的概率正交. 为了控制维度灾难, 一个简单的方法是使用每一个字节作为一个词统计. 这个模型虽然看上去很没道理, 而且还很”面向机器”, 但是考虑到计算机处理语言的时候本来也就是根据字节来处理, 这样做一下也未尝不可.

我并没有亲自实践比较字节频率然后计算向量距离的方法, 不过我可以想像, 这样的方法在UTF-8字符集上是有很好表现的. 有人可能会担心, 加入某个汉字编码成了两个字节, 这两个字节正好分别等于西文字母的编码的话, 汉字就可以在计算机中等价的写成一串字母了, 这样, 就很难保证一段汉字是不是和一段希腊文正好有同样的字节分布了, 因为我们可以作弊一下, 搞出一段汉字, 正好和希腊文对应分布一样. 但是, 巧妙之处在于UTF-8 字符集本身通过若干前缀区分了不同的变长编码, 不同的编码区段没有任何交集. 反过来说就是, 从UTF-8 编码的字节序列中任意取一个字节出来, 一定能够区分这个字节是某个变长编码的一部分. 任何一个多字节符号编出来的编码的一部分, 绝对不可能正好是其他单字节符号的编码. 这样, 单字节多字节变成字节编码后各自为政, 映射到的字节码的概率分布不会出现“作弊”现象. 用严格的数学语言来说就是, 一个字符对应的字节在叠加到向量中的时候, 绝对不会叠加到属于另一个字符的所有字节上, 也就是说, 他们总有独立分布的部分存在. 这个是字节模型能正确反映语言模型的一个必要要求.

[To be continued…]

Comments (1)

识别文本用哪种语言写成

ASPN Python Cookbook 提到了一个使用 zlib 库识别文本用哪种语言写成的程序. 其核心代码不超过20行, 据我观察, 识别精度不低于95%. 我略做了一下修改, 把联合国联合国人权宣言作为语料库,目前从 wikipedia 上随便抓一篇爪哇文的文章下来, 都能识别得九不离十。

class Entropy:
    def __init__(self):
		self.entro = []

    def register(self, name, corpus):
        “”"
        register a text as corpus for a language or author.
        <name> may also be a function or whatever you need
        to handle the result.
        “”"
        corpus = str(corpus)
        ziplen = len(zlib.compress(corpus))
        print name, ziplen
	self.entro.append((name, corpus, ziplen))

    def guess(self, part):
        “”"
        <part> is a text that will be compared with the registered
        corpora and the function will return what you defined as
        <name> in the registration process.
        “”"
        what = None
        diff = 0
        part = str(part)

        for name, corpus, ziplen in self.entro:
		nz = len(zlib.compress(corpus+part)) - ziplen
		if diff==0 or nz<diff:
                	what = name
        		diff = nz
        return what

先贴代码, 有时间细讲一下语言模型和信息论的妙用. 简单而小巧的模型解决看上去不可解决的问题, 这就是人工智能的精华.

[所有文件打包下载(包含语料源文件10Mb). 代码本身其实只有50行]

Comments (1)

A Practical Guide to the AIMA Python Code 2

Continued with the last post

Some readers reported that if using graph_search to solve the NQueensProblem provided in search.py, python will complain that:

TypeError: list objects are unhashable

Yes, you are doing perfectly right and the code itself is imperfect. As this point, you might want to make the List objective hashable. Inspired by the idea of Set and FrozenSet in Python, you would expect a frozen(x) function to make any mutable object immutable (and therefore hashable) [Or you can try to use tuple from beginning, but in the NQueensProblem code, it uses .index method, which is only suitable for List]. Unfortunately, this proposal (PEP-351) was rejected by Guido. Hence, we don’t have a unified way to convert any “state” to an immutable object and insert it in to “closed table”. One possible way to do this is to write your own State class and implement the __hashcode__ function. The other handy but awkward way is to convert the list to a tuple when insterting to the “closed table, i.e. edit Line 144 of search.py as:

if tuple(node.state) not in closed:
closed[tuple(node.state)] = True

Here is a toy forzen function.

“”"Toy function frozen which can convert mutable objects to immutable
	Author: Eric You XU.
	GPLv2
“”"

def frozen(x):
	“”"Return the immutable version of an object.
	“”"
	if type(x) == type([]):
		return tuple(x)
	elif type(x) == type([]):
		raise “I can’t froze dictionary object as they are very hot.”
	# fill your type here
	else:
		return x

You can put it in to search.py and edit the Line 144 as:

if frozen(node.state) not in closed:
colsed[
frozen(node.state)] = True

Now the code looks much better.

Comments

A practical guide to the AIMA Python code -1

Part -1 Search

Introduction

images.jpg AIMA (Artificial Intelligence: A Modern Approach) might be the most successful textbook about AI. I take this course with Prof. Yixin Chen this semester. I am going to go through all the source code (Python) provided on the website and use them to finish my homework. Therefore, both for my personal referral purpose and for someone who might be interested in using that code, I write a very simple “practical guide to AIMA python source code” series here. Enjoy the magic of AI and Python!
Step 1, build the environment.

In this section, you will need three python files:

http://aima-python.googlecode.com/svn/trunk/agents.py
http://aima-python.googlecode.com/svn/trunk/search.py
http://aima-python.googlecode.com/svn/trunk/utils.py

Type:
python agents.py
and
python search.py

to make sure that you have all the packages like Image and Tk with your python. If you don’t have them, just download them. The general guideline is to use apt-get if possible. If you are not using Debian family OS, download the Python package and then use

python setup.py

at the python package folder.

Step 2, study the code — tree search

In order to use the code, we have to get familiar with the interface. Let’s get start from search.py and start from depth_first_tree_search (Line 130). The line reads:

def depth_first_tree_search(problem):
return tree_search(problem, Stack())

We all know that depth-first search uses Stack. Now let’s omit the technical detail about the implementation of Stack in util.py and check what the “problem” is.

Let’s move to Line 16 of search.py. Here is the definition of Problem class. Here __init__ just assigns the initial state. All functions like successor and goal_test are abstract. So, let’s take a look a concrete example. Let’s jump to Line 458, NQueenProblem. It’s a subclass of Problem. It defines the initial state, successor function, and goal_test function. Node that the function conflicted and conflict are for internal use only, there are not the essential part of a problem.

Here, by this case study, we at least know that we have to define a problem with

1. self.initial field which is the starting point for a problem.
2. A successor function which generate the successor state for a given state. We will study it carefully later.
3. A goal_test function which returns either False or True.

The initial is really trivial, we can just assign a initial state to it. Note that Python is a dynamic type language, so we don’t actually care about what type of state is. We will exam the type of state later, but now let’s assume that we know the type of state.

The successor function is really tricky. A straight-forward way to design a successor function will return all possible states in a list. But if you do that, you will get an “Too many values to unpack” error. The design here is actually very neat: instead of put a state in to the list, you can put a (action, state) tuple in to a list. [Line 88 of search.py, in

function expand(problem)
...
for (act, next) in problem.successor(self.state)

read this line and you will figure out the date structure to store the successors. The reason is also simple: we can trace the path easily if you record the action down at every expanding step.

Since we moved our eyes to the Node class, let’s exam it carefully.

The constructor of Node is fairly simple. The path function will backtrack all the path to the root. We can modify the code of __repr__ here to print the action out if needed. If any search finally return a result in the form of Node, we can actually get everything we need from this Node, e.g., the path to the node, the action series to the node.

Based on the knowledge above, you can write your own problem model based on tree search. Now let’s move on to graph search.

Step 3. Graph search.

The only difference between tree search and graph search is that in graph search, we have a fringe that stores the closed notes. As you might know, the fringe is implemented in hash table. In Python, there is a tiny technical point: List is not hash-able as it’s mutable, Tuple and String are hash-able. Therefore, if you implement your state as a list, make sure to convert it to a tuple or string or a constant. Otherwise graph search will not be happy with this and complain about it.

If you want to use A* search, another point arises: heuristic function. Note that Line 211 reads:

def f(n):
return max(getattr(n, 'f', -infinity), n.path_cost + h(n))

here n is a Node instead of a State. Hence, if you want to get the state information and implement h, you have to use n.state to get the state information.

Step 4. build your application

Perhaps the best part to study the code is to write your own code to test it. Here are my implementations on eightpuzzle and three-missionaries three cannibals problem. The code is ugly and only for study purpose only. Please don’t directly copy it to your homework.

Eightpuzzle.py

#!/usr/bin/python

from utils import *;
from search import *;

class EightPuzzle(Problem):
	“”"The problem of solving a eight puzzle problem “”"
	def __init__(self, state):
		self.initial = tuple(state)

	def successor(self, state):
		“”"In the 8 puzzle problem, move the blank to other position.”"”
		result = []
		position = list(state).index(0)
		for action, newposition in self.possible_position(position).iteritems():
			new = list(state[:])
			new[newposition], new[position] = new[position], new[newposition]
			result.append((action, tuple(new)))
		return result

	def possible_position(self, position):
		“”"return the possible position”"”
		result = {};
		col = position % 3;
		row = position // 3;
		if row != 2:
			result[‘D’] = position +3
		if col != 0:
			result[‘L’] = position -1
		if row != 0:
			result[‘U’] = position -3
		if col != 2:
			result[‘R’] = position +1

		return result
	

	def goal_test(self, state):
		if state[0] != 0:
			return False
		elif state[1] !=1:
			return False
		elif state[2] !=2:
			return False
		elif state[3] !=3:
			return False
		elif state[4] !=4:
			return False
		elif state[5] !=5:
			return False
		elif state[6] !=6:
			return False
		elif state[7] !=7:
			return False
		elif state[8] !=8:
			return False
		else:
			return True 	

	def h(self, node):
		“”"Heuristic function”"”
		temp=list(node.state)
		sum=0
		for x in range(8):	# mahatten distance
			sum = sum + abs(temp.index(x)-x)
		return sum

	def h2(self, node):
		“”"bad h”"”
		sum = 0
		pos = 0
		for x in node.state:
			sum = sum + abs(x - pos)
			pos = pos + 1
		return sum
if (__name__==“__main__”):

#	state = [1, 2, 3, 4, 5, 6, 7, 8, 0]
	state = [7, 2, 4, 5, 0, 6, 8, 3, 1]
#	state = (2, 6, 4, 7, 0, 3, 5, 8, 1)
	instance = EightPuzzle(state)
#	breadth_first_tree_search(instance)
#	depth_first_tree_search(instance)
#	breadth_first_graph_search(instance)
#	print depth_first_graph_search(instance).path()
	print astar_search(instance, instance.h).path() 

Boat.py

#!/usr/bin/python

from utils import *;
from search import *;

“”"state is a tuple with (#m1, #m2, #s1, #s2, where_is_the_boat)”"”

class Boat(Problem):
	def __init__(self, state):
		self.initial = tuple(state)
		self.N = state[0]
		self.nodenum = 1

	def successor(self, state):
		result = []
		symbol = state[4]*2 -1; # 0 => -1
		action = 0
		for x in (0, 1, 2):
			for y in (0, 1, 2):
				if x + y >= 1 and x + y <= 2:
					new = (state[0] + symbol*x, state[1]-symbol*x, state[2] + symbol*y, state[3] - symbol*y, 1 - state[4])
					if self.check(new):
						result.append((action, new))
						action = action + 1
		self.nodenum = self.nodenum + action
		print result
		return result

	def check(self, state):
		for x in state[:-1]:
			if x<0:
				return False;

		if state[0]!=0 and state[0] < state[2]: return False;
		if state[1]!=0 and state[1] < state[3]: return False;

		return True

	def goal_test(self, state):
		if state[0] == 0 and state[1] == self.N and state[2] == 0 and state[3] == self.N and state[4] ==1:
			print state
			return True
		return False;

	def h(self, node):
		return 0
		state=node.state
		return state[0]

if (__name__==“__main__”):

	state=[3, 0, 3, 0, 0]

	instance = Boat(state)
#	breadth_first_tree_search(instance)
#	depth_first_tree_search(instance)
#	breadth_first_graph_search(instance)
#	depth_first_graph_search(instance)
	astar_search(instance, instance.h)
	print instance.nodenum

BTW, if you need to highlight your source code and convert it to HTML for example, one of the handy program is GNU source-highlight. I really love this.

Comments (1)