UTF-8 with guile
2008/11/19 @ 23:03Getting UTF-8 to work with guile is a bit of a stretch as guile doesn't have any real encoding or UTF-8 support to speak of, but I was able to get at least some basic stuff working by using the Unicode Manipulation routines which are part of the Guile-Glib module.
This requires that you have your LD_LIBRARY_PATH and GUILE_LOAD_PATH set properly so that it can load the glib libraries. This didn't work for me out of the box with guile, but Ubuntu's guile-gnome0-glib package provides a script called guile-gnome-0 for setting these values for you and running guile. If you just run guile as is...
ian@laptop:~# guile
guile> (use-modules (gnome glib))
Backtrace:
In standard input:
1: 0* (use-modules (gnome glib))
1: 1 (eval-case (# # *unspecified*) (else #))
1: 2 (begin (process-use-modules (list (list #))) *unspecified*)
In unknown file:
?: 3* [process-use-modules (((gnome glib)))]
<unnamed port>: In procedure process-use-modules in expression (process-use-modules (list #)):
<unnamed port>: no code for module (gnome glib)
ABORT: (misc-error)
guile>
Anyway you can run guile with the right paths on Ubuntu by running guile-gnome-0. After getting that running I made use of the Guile-Glib module to do some operations of utf8 data. I did this from a utf8 console. Beware. The terminal was not to friendly with me about things like arrow keys and backspaces over multibyte characters etc.
guile> (use-modules (gnome glib))
WARNING: (gnome gw generics): imported module (gnome gobject generics) overrides core binding `connect'
guile> (string-length "テスト")
9
guile> (g-utf8-strlen "テスト" -1)
3
guile> (g-utf8-validate "テスト" -1 #f)
#t
guile> (g-unichar-validate (g-utf8-get-char "手"))
#t
Django Sitemap Framework
2008/11/18 @ 21:22Using the Django sitemap framework is so easy it's almost no work at all. Just make a sitemap object and add it to the sitemap in urls.py. The sitemap framework calls items() in your Sitemap to get the list of objects to put in the sitemap and then calls get_absolute_url() on each object.
models.py
| from django.db import models | |
| ... | |
| class Entry(models.Model): | |
| ... | |
| @permalink | |
| def get_absolute_url(self): | |
| return ... | |
| ... |
sitemap.py
| from django.contrib.sitemaps import Sitemap | |
| from mysite.blog.models import Entry | |
| from django.contrib.sitemaps import Sitemap | |
| from mysite.blog.models import Entry | |
| class BlogSitemap(Sitemap): | |
| priority = 0.5 | |
| def items(self): | |
| return Entry.objects.filter(is_draft=False) | |
| def lastmod(self, obj): | |
| return obj.pub_date | |
| # changefreq can be callable too | |
| def changefreq(self, obj): | |
| return "daily" if obj.comments_open() else "never" |
urls.py
| from mysite.blog.sitemap import BlogSitemap | |
| ... | |
| sitemaps = { | |
| "blog": BlogSitemap | |
| } | |
| (r'^sitemap.xml$', 'django.contrib.sitemaps.views.sitemap', {'sitemaps': sitemaps}) | |
| ... |
You can even generate sitemap indexes and it will pagenate the indexes on Google's limit of 50,000 urls so that you don't have a problem with it crawling your indexes.
New Apartment
2008/11/15 @ 23:22Recently I've been writing a lot of blog posts about programming topics and I haven't really talked about much of the many things that have happened recently. Two big things and many many small things have happened recently. I'll start with the big things and perhaps blog about some of the smaller things at a later time.
Big thing #1: I got a new job.
I had been looking for a new job for a while but without luck but a good opportunity come to me in September so I've moved from my job programming mostly SOAP web service backend server applications at Think to a new job programming Web applications in Python using Django at BeProud. This might be Greek to a lot of you but the end result is that work is a lot more fun now. I am also getting paid about twice as much as before to do it.
Another big difference is that there are no English speaking folks at the new job. This might seem seem like it would be a con rather than a pro but it means that I no longer have to support other folks because they don't speak Japanese. This makes work a whole lot easier. About 25% of my previous job was to support other programmers who don't speak Japanese and aren't familar with Japanese business practices and customs. This meant training and teaching. It also meant that for every meeting I attended in Japanese I would either had to have another meeting in English to inform the non-Japanese speaking folks what was decided and why, or no one would know what was going on. Not having to worry about that makes a big difference.
Big thing #2: I moved to a new apartment.
With my new job being in a different place and since my old apartment was in my old company's name I moved at the beginning of this month. The new apartment is over twice as big ( and twice as expensive) but it was worth it. The old apartment was tiny. And I mean tiny. In Japanese the apartment not including the kitchen and bathroom was 6 Jyou. Which is roughly 10m2.Which is roughly 107 square feet. The total apartment was probably about 125 square feet or so. So I'm not joking when I say it's tiny. And I lived there for 2 years. Even the new apartment is only about 450 square feet but by Tokyo standards it's large for a single person. In any case the new place is much easier to live in.
Here are some pictures.
Django admin inline forms
2008/11/09 @ 01:09For my new project dlife, I went about implementing a simple comments interface that would allow users to make comments on imported feed items. I wanted to support this in the admin in the typical manner such that when you click on an item in the admin, you can see all the comments and edit them from the item's page.
I found that you can use inline forms in the admin but it seems to show a bunch of forms (3 by default) even though I don't have any comments for the item yet. I'll mess with this a bit more later to try to get the behavior I want.
models.py
| class Comment(models.Model): | |
| '''An item comment''' | |
| comment_item = models.ForeignKey(Item) | |
| comment_date = models.DateTimeField() | |
| comment_user = models.ForeignKey(User, null=True, blank=True) | |
| comment_name = models.CharField(max_length=30) | |
| comment_email = models.EmailField() | |
| comment_homepage = models.URLField(max_length=300) | |
| comment_content = models.TextField(null=True, blank=True) | |
| | |
| class Meta: | |
| db_table="comments" | |
| ordering=["comment_item", "-comment_date"] |
admin.py
| class CommentInline(admin.StackedInline): | |
| model = Comment | |
| max_num = 1 #TODO: Fix this | |
| exclude = ['comment_item','content_type','object_id'] | |
| class ItemAdmin(admin.ModelAdmin): | |
| list_display = ('item_title', 'item_date') | |
| exclude = ['item_clean_content',] | |
| list_filter = ('item_feed',) | |
| search_fields = ('item_title','item_clean_content') | |
| list_per_page = 20 | |
| | |
| inlines = [CommentInline,] |
Feedparser and Django
2008/10/30 @ 19:13Over the weekend at Python Onsen I worked on a lifestream web application using Django and feedparser. I was really impressed with how simple feedparser is to use and how easy it is to get unified results from atom or rss feeds. You simply import feedparser and call feedparser.parse to parse a feed from a url.
feeds.py| ... | |
| def update_feeds(): | |
| feeds = Feed.objects.filter(feed_deleted=False) | |
| for feed in feeds: | |
| try: | |
| feed_items = feedparser.parse(feed.feed_url) | |
| for entry in feed_items['entries']: | |
| ... |
You can check out feeds.py here.
The interesting bit comes with how I had to parse the dates which sometimes include timezone info and other goodies. In my search for a solution to the problem of how to deal with dates in various formats I turned came across this blog entry which describes the problem and some possible solutions. The solution I used was the simplest and most robust (please skip the comments talking about taking a slice of the date string). I used mikael's suggestion from the comments and used the dateutil.parser to parse the date string into a proper datetime object.
| # Parse to an actual datetime object | |
| date_published = dateutil.parser.parse(date_published) |
This however leaves timezone info on the timestamp which isn't supported by mysql so I hand rolled some code convert the timestamp to utc and remove the timezone info.
| # Change the date to UTC and remove timezone info since MySQL doesn't | |
| # support it | |
| date_published = (date_published - date_published.utcoffset()).replace(tzinfo=None) |
I'm not sure this works in all situations yet so I might go with something like how another commenter solved the problem by converting feedparsers parsed date to a utc timestamp before converting to a datetime object. I think either way would work but which is cleaner and less prone to breakage, I'm not sure.
Python Onsen Oct. 2008
2008/10/29 @ 17:12Last weekend I went to my second Python Onsen[jp] organized by Nakai-san(id:voluntas). I talked about Python Onsen in my first blog post here. Python Onsen is a 3 day event (Fri, Sat, Sun) but as before I only participated on Saturday and Sunday. This time I opted to work on creating a lifestream web app using feedparser and Django. feedparser is a snappy little parser for reading RSS and Atom feeds. The result was dlife which so far can parse a set of feeds and show them on a user's lifestream though it's not in any way user friendly yet (you have to update the feeds in the django shell 
).
I also got to know my soon to be co-worker, Okano-san (id:tokibito[jp]), by talking about jQuery Internals' data() function and web/Django development.
Here's a recap:
- Worked on a sweetcron lifestream replacement in Django (dlife)
- Onsen is pretty lonely by yourself.
- Introduced id:tokibito[jp] to the jQuery Internals' data() function
- Since I come on Satruday, I always miss introductions so I never know who is who.
- feedparser is really simple and easy to use. Though I'm not sure what I'll do about pictures and video yet.
- No one mentioned my blog or linked to it in their posts
(Maybe because I never write anything? ) - In a raffle、I got a cool Python shirt from Accense Technologies'[jp] Masuda-san(id:whosaysni[jp]).
- It was lonely going home by myself from Kinomiya station.
XML Feeds
