django haystack index update only new items

When solr index becomes bigger, update_index is taking some time and in this period search is returning empty resultset to prevent this basicly you only need to include to the index latest modified records in database.

following changes in your model and index structure help you to achieve fast index updates


models.py
class MyModel(models.Model):
    user = models.ForeignKey(User)
    modified = models.DateTimeField(auto_now=True) # add this field to your model
search_indexes.py
class MyModelIndex(SearchIndex):
    text = CharField(document=True, use_template=True)
    user = CharField(model_attr='user')

    # add this function to your search_indexes.py
    def get_updated_field(self):
        return "modified"
./manage.py update_index --age=110 
indexing will run only items updated in the last 110 hours.

Comments

Post a Comment

Popular posts from this blog

Pyppeteer fix for BrowserError: Browser closed unexpectedly

if you are having an error related with Pyppeteer   BrowserError: Browser closed unexpectedly   is just the error you get when Chrome crashes for whatever reason. It would be nice if pyppeteer printed out the error, but it doesn't. To track things down, it's helpful to pull up the exact command that pyppeteer runs. You can do that this way: from pyppeteer.launcher import Launcher ' '.join(Launcher().cmd) run output from command line and find missing libraries and install it.
Read more

overlay filesystem and containers

Containers A  container  is a process running on the host system.  This process is started by the container runtime from a container image, and the runtime provides a set of tools to manage the container process.  Namespaces virtualize the container process’s PID, network, root, and users.  Cgroups help set resource usage limits the container process can consume on the host system, and security contexts enforce permissions the container process has on the host system. A container, as a runtime object, consumes the typical resources any running process would consume on a system: storage for the file system and any saved configuration files, CPU, memory, and networking to serve traffic to/from external clients, and other containers or devices on the system. docker run container from image and connect it with shell , install necessary networking tools docker run -d -it ubuntu:22.10 bash docker run -d -it --privileged ubuntu:22.10 bash -> if you need to write to filesystem run wit priv
Read more

How to add pagination to django comments for your model

First step is import Comment and ContentType to your view like below from django.contrib.comments.models import Comment from django.contrib.contenttypes.models import ContentType Second step is defining your model which you want to paginate its comments content_type_id= ContentType.objects.get_for_model( YOUR_MODEL_NAME ) Last get comments of specific instance of your model class with defining object_pk as filter parameter comments = Comment.objects.filter(content_type=content_type_id,object_pk=quote_id) here is complete view code from django.core.paginator import Paginator, InvalidPage, EmptyPage from django.contrib.comments.models import Comment from django.contrib.contenttypes.models import ContentType def quote_show_page(request,quote_id): quote = get_object_or_404(Quote, id=quote_id) content_type_id=ContentType.objects.get_for_model(Quote) comments = Comment.objects.filter(content_type=content_type_id,object_pk=quote_id) paginator = Paginator(comments, 5) # Show 5 comments pe
Read more