... a command line tool to monitor your sitemap links

I was thinking to make such tool for a while and fortunately I found some time so here it is.

Simple Site Checker is a command line tool that allows you to run a check over the links in you XML sitemap.

How it works: The script requires a single attribute - a URL or relative/absolute path to xml-sitemap. It loads the XML, reads all loc-tags in it and start checking the links in them one by one.
By default you will see no output unless there is an error - the script is unable to load the sitemap or any link check fails.
Using the verbosity argument you can control the output, if you need more detailed information like elapsed time, checked links etc.
You can run this script through a cron-like tool and get an e-mail in case of error.

I will appreciate any user input and ideas so feel free to comment.

... or how to imitate dynamic properties in a class object

Preface: When you have connections between your application and other systems frequently the data is not in the most useful form for your needs. If you have an API it is awesome but sometimes it just does not act the way you want and your code quickly becomes a series of repeating API calls like api.get_product_property(product_id, property).
Of course it will be easier if you can use objects to represent the data in you code so you can create something like a proxy class to this API:

class Product(object):
    def __init__(self, product_id):
        self.id = product_id

    @property
    def name(self):
        return api_obj.get_product_property(self.id, 'name')

    @property
    def price(self):
        return api_obj.get_product_property(self.id, 'price')

#usage
product = Product(product_id)
print product.name

In my opinion it is cleaner, more pretty and more useful than the direct API calls. But still there is something not quite right. Problem: Your model have not two but twenty properties. Defining 20 method makes the code look not that good. Not to mention that amending the code every time when you need a new property is quite boring. So is there a better way? As I mention at the end of Connecting Django Models with outer applications if you have a class that plays the role of a proxy to another API or other data it may be easier to overwrite the __getattr__ method. Solution:

class Product(object):
    def __init__(self, product_id):
        self.id = product_id

    def __getattr__(self, key):
        return api_obj.get_product_property(self.id, key)

#usage
product = Product(product_id)
print product.name

Now you can directly use the product properties as attribute names of the Product class. Depending from the way that the API works it would be good to raise AttributeError if there is no such property for the product.

I am starting a new blog. The reason is that I want to keep this one more technically oriented while the other will be more business and customers oriented. Its name is Software for business and the idea is to show to the business in less technical details how the modern IT technologies like CRM & ERP systems, web sites, e-commerce solutions, online marketing and so on can help their business.
If you find the topic interesting feel free to join.

Preface: Sometimes, parts of the data that you have to display in your application reside out of the Django models. Simple example for this is the following case - the client requires that you build them a webshop but they already have CRM solution that holds their products info. Of course they provide you with a mechanism to read this data from their CRM.

Specialty: The problem is that the data in their CRM does not hold some of the product information that you need. For instance it misses SEO-friendly description and product image. So you will have to set up a model at your side and store these images there. It is easy to join them, the only thing that you will need is a simple unique key for every product.

Solution: Here we use the product_id field to make the connection between the CRM data and the Django model.

# in models.py
class Product(models.Model):
    product_id = models.IntegerField(_('Original Product'),
                                     unique=True)
    description = models.TextField(_('SEO-friendly Description'),
                                   blank=True)
    pod_image = FilerImageField(verbose_name=_('Product Image'),
                                blank=True, null=True)

   @property
   def name(self):
       return crm_api.get_product_name(self.product_id)

# in forms.py
class ProductForm(forms.ModelForm):
    name = forms.CharField(required=False,
                           widget=forms.TextInput(attrs={
                               'readonly': True,
                               'style': 'border: none'}))

    class Meta:
        model = Product
        widgets = {
            'product_id': forms.Select(),
        }
    def __init__(self, *args, **kwargs):
        super(ProductForm, self).__init__(*args, **kwargs)
        self.fields['product_id'].widget.choices = crm_api.get_product_choices()
        if self.instance.id:
            self.fields['name'].initial = self.instance.name

The form here should be used in the admin(add/edit) page of the model. We define that the product_id field will use the select widget and we use a method that connect to the CRM and returns the product choices list.
The "self.instance.id" check is used to fill the name field for product that are already saved.

Final words: This is a very simple example but its idea is to show the basic way to connect your models with another app. I strongly recommend you to use caching if your CRM data is not modified very often in order to save some bandwidth and to speed up your application.
Also if you have multiple field it may be better to overwrite the __getattr__ method instead of defining separate one for each property that you need to pull from the outer application.

P.S. Thanks to Miga for the code error report.

... no problem cause we have stat_float_times

Preface: recently I downloaded PyTDDMon on my office station and what a surprise - the files changes did not affect on the monitor. So I run a quick debug to check what is going on: the monitor was working running a check on every 5 seconds as documented but it does not refresh on file changes.

Problem: it appeared that the hashing algorithm always returns the same value no matter whether the files are changed or not. The calculation is based on three factors: file path, file size and time of last modification. I was changing a single file so the problem was in the last one. Another quick check and another surprise - os.stat(filename).st_mtime returns different value after the files is changed but the total hash stays unchanged. The problem was caused from st_mtime returning floating point value instead of integer.

Solution: Unfortunately the os.stat results appeared to be OS specific. Fortunately the type of the returned result is configurable. Calling os.stat_float_times(False) in the main module force it to return integer values.

Final words: these OS/version/settings things can always surprise you. And most of the time it is not a pleasant surprise. Unfortunately testing the code on hundred of different workstations is not an option. But this is one of the good sides of the open source software. You download it, test it, debug it and have fun.
Also special thanks to Olof(the man behind the pytddmon concept) for the tool, the quick response to my messages and for putting my fix so fast in the repo. I hope that this has saved someone nerves )

P.S. If you are using Python and like TDD - PYTDDMON is the right tool for you.