RSS

Monthly Archives: November 2015

Getting Started With RabbitMQ in .net

By: Simon Dixon’s Blog

In the previous two examples I built a simple .net application to demonstrate first two sections the RabbitMQ getting started guide in .net. In this post I’ll be looking at the third. Download the Source

3.) Publish/Subscribe

The original article (in Java) is here:http://www.rabbitmq.com/tutorials/tutorial-three-java.html

I’m going to take a slightly different approach to my previous two examples and split the Producer and Consumer into two different Windows Forms. This will allow us to run as many Consumers as we like  and so demonstrate Pub/Sub effectively.

First up is the Producer.

Create a new Form and add the Input TextBox and Button as in the first two examples. Also and a new Button “Start New Consumer” .

Next create the consumer. We only need to output messages so we only need one RichTextBox

In the previous two examples we had pretty much duplicate constructors for both Consumers and Producers. We will now fix this by creating a base class that these can both inherit from. Create a new class called IConnectToRabbitMQ.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
public abstract class IConnectToRabbitMQ : IDisposable
    {
        protected IModel Model { get; set; }
        protected IConnection Connection { get; set; }
        public string Server { get; set; }
        public string ExchangeName{ get; set; }
        public string ExchangeTypeName { get; set; }
        public IConnectToRabbitMQ(string server, string exchange, string exchangeType)
        {
            Server = server;
            Exchange = exchange;
            ExchangeTypeName = exchangeType;
        }
        //Create the connection, Model and Exchange(if one is required)
        public virtual bool ConnectToRabbitMQ()
        {
            try
            {
                var connectionFactory = new ConnectionFactory();
                connectionFactory.HostName = Server;
                Connection = connectionFactory.CreateConnection();
                Model = Connection.CreateModel();
                bool durable = true;
                if (!String.IsNullOrEmpty(Exchange))
                    Model.ExchangeDeclare(Exchange, ExchangeTypeName, durable);
                return true;
            }
            catch (BrokerUnreachableException e)
            {
                return false;
            }
        }
        public void Dispose()
        {
            if (Connection != null)
                Connection.Close();
            if (Model != null)
                Model.Abort();
        }
    }

The class name may look a little odd to most as it begins with an “I”, this is usually the naming convention for an Interface but I’m using what I like to call Simon Says naming convention. I’ll be writing a post about this in the near future. The main gist of it, is I like to have classes tell me what they do. For example, a class which calls a remote service might inherit from a class(or interface) called ICallRemoteServices. So the full class name definition would be FooService : ICallRemoteServices. There would also be a abstract method defined that implements the action e.g CallRemoteService.  Other example are  IAmAnOrder(for a value object), ICalculateShipping, IDeliverEmail etc. This may seem a little weird but I like it 🙂.

So enough of that for now, lets go through the class. First we declare fields to hold the familiar IModeland Connection instances. Next up are fields to store the details of the Server, Exchange andExchangeTypeName.  Exchange is the name of the exchange we want to publish/consume messages from and ExchangeTypeName  holds the type of exchange we want to use(in this example it will be “fanout”).  ExchangeType is set from a constant declared in the RabbitMQ.Client.ExchangeType  class, so for us it will be ExchangeType .Fanout(More on this later.) Next we have the  ConnectToRabbitMQ() method, this is almost exactly the same as the Constructor methods of the Producer/Consumer methods  in my  previous two examples. We have this additional block which declares the Exchange.

1
2
3
bool durable = true;
if (!String.IsNullOrEmpty(ExchangeName))
    Model.ExchangeDeclare(ExchangeName, ExchangeTypeName, durable);

We are declaring a durable exchange of the type ExchangeTypeName with the name ExchangeName.  If this exchange had already been declared by another Producer or Consumer  a new one is not created, the existing one will be used.

Now we’ll write our Producer.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
public class Producer : IConnectToRabbitMQ
    {
        public Producer(string server, string exchange, string exchangeType) : base(server, exchange, exchangeType)
        {
        }
        public void SendMessage(byte[] message)
        {
            IBasicProperties basicProperties = Model.CreateBasicProperties();
            basicProperties.SetPersistent(true);
            Model.BasicPublish(ExchangeName, "", basicProperties, message);
        }
    }
}

Here we a have a nice lightweight publisher, the only difference from our previous examples is we are publishing to a named exchange called ExchangeName. We do not know about or use a Queue.

Then it’s our consumer, this is slightly more complicated.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
public class Consumer : IConnectToRabbitMQ
    {
        protected bool isConsuming;
        protected string QueueName;
        // used to pass messages back to UI for processing
        public delegate void onReceiveMessage(byte[] message);
        public event onReceiveMessage onMessageReceived;
        public Consumer(string server, string exchange, string exchangeType) : base(server, exchange, exchangeType)
        {
        }
        //internal delegate to run the consuming queue on a seperate thread
        private delegate void ConsumeDelegate();
        public void StartConsuming()
        {
                Model.BasicQos(0, 1, false);
                QueueName = Model.QueueDeclare();
                Model.QueueBind(QueueName, ExchangeName, "");
                isConsuming = true;
                ConsumeDelegate c = new ConsumeDelegate(Consume);
                c.BeginInvoke(null, null);
        }
        protected Subscription mSubscription { get; set; }
        private void Consume()
        {
            bool autoAck = false;
            //create a subscription
            mSubscription = new Subscription(Model, QueueName, autoAck);
            while (isConsuming)
            {
                BasicDeliverEventArgs e = mSubscription.Next();
                byte[] body = e.Body;
                onMessageReceived(body);
                mSubscription.Ack(e);
            }
        }
        public void Dispose()
        {
            isConsuming = false;
            base.Dispose();
        }
    }

We need to store the name of our Queue that we will be binding to the exchange so we have a field QueueName for this purpose. The next code of interest is the StartConsuming() method. Most of this is familiar with this additional block:

1
2
QueueName = Model.QueueDeclare();
Model.QueueBind(QueueName, ExchangeName, "");

What we are doing here is asking the model to declare a temporary queue for us and give it a random unique name(stored in QueueName), we then bind this queue to the Exchange called ExchangeName. 

This a key concept to exchanges in RabbitMQ, a publisher/producer only knows about the Exchange, it will publish messages directly to the Exchange and has no concept of a queue. Each consumer knows about the Exchange but they will also have a queue that is bound to the Exchange.  The way I look at it is the one or more Producers own an Exchange(and publish to it) and each Consumer owns a Queue(which is bound to an Exchange.)

The Consume() method is very different to what we have seen before(and the Java Example). Instead of using a  QueueingBasicConsumer we are using a Subscription. Subscription is part of theRabbitMQ.Client.MessagePatterns package in the .net client Library. It give us a nice wrapper to the boilerplate message de-queuing code. More info is here.

1
2
3
4
5
mSubscription = new Subscription(Model, QueueName, autoAck);
  .....
  BasicDeliverEventArgs e = mSubscription.Next();
  .....
  mSubscription.Ack(e);

Now we need to add the code for our Producer Form

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
public string HOST_NAME = "localhost";
 public string EXCHANGE_NAME = "logs";
 private Producer producer;
 //delegate to show messages on the UI thread
 private delegate void showMessageDelegate(string message);
 public PubSub_Producer()
 {
     InitializeComponent();
     //Declare the producer
     producer = new Producer(HOST_NAME, EXCHANGE_NAME, ExchangeType.Fanout);
     //connect to RabbitMQ
     if(!producer.ConnectToRabbitMQ())
     {
         //Show a basic error if we fail
         MessageBox.Show("Could not connect to Broker");
     }
 }
 private int count = 0;
 private void button1_Click(object sender, EventArgs e)
 {
     string message = String.Format("{0} - {1}", count++, textBox1.Text);
     producer.SendMessage(System.Text.Encoding.UTF8.GetBytes(message));
 }
 private void button2_Click(object sender, EventArgs e)
 {
     //Open a new Consumer Form
     PubSub_Consumer consumer = new PubSub_Consumer();
     consumer.Show();
 }

This should be fairly self explanatory. The producer.ConnectToRabbitMQ() call is handled in the base IConnectToRabbitMQ class. We’ve added little error handling code just in case the broker is unavailable(if it is run rabbitmq-server -detached from the command line .) There’s also a method to handle clicks on the “Start New Consumer” Button which spawns a new Consumer Form.

Then we have our Consumer Form.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
public partial class PubSub_Consumer : Form
    {
        public string HOST_NAME = "localhost";
        public string EXCHANGE_NAME = "logs";
        private Consumer consumer;
        public PubSub_Consumer()
        {
            InitializeComponent();
            //create the consumer
            consumer = new Consumer(HOST_NAME, EXCHANGE_NAME, ExchangeType.Fanout);
            //connect to RabbitMQ
            if (!consumer.ConnectToRabbitMQ())
            {
                //Show a basic error if we fail
                MessageBox.Show("Could not connect to Broker");
            }
            //Register for message event
            consumer.onMessageReceived += handleMessage;
            //Start consuming
            consumer.StartConsuming();
        }
        //delegate to post to UI thread
        private delegate void showMessageDelegate(string message);
        //Callback for message receive
        public void handleMessage(byte[] message)
        {
            showMessageDelegate s = new showMessageDelegate(richTextBox1.AppendText);
            this.Invoke(s, System.Text.Encoding.UTF8.GetString(message) + Environment.NewLine);
        }
    }

This is exactly the same as previous Consumer examples with the additional call to the base class.

Now we can run the project after making sure the correct Form is opened on starup

1
2
3
4
5
6
7
[STAThread]
     static void Main()
     {
         Application.EnableVisualStyles();
         Application.SetCompatibleTextRenderingDefault(false);
         Application.Run(new PubSub_Producer());
     }

Click the “Start New Consumer” Button a couple of time to get a few consumers running, then put  your message in the “Producer Input”  TextBox and hit send. You should see the message appear in all the Consumer output windows. Good stuff 🙂

Summary

What we have done here is create a Fanout Exchange named “logs”,  we’ve created some Consumers(three in my example above) each with their own unique temporary queue bound to the exchange. We have then published a message to the exchange using our Producer, the exchange then routes the message to all bound queues which in turn delivers it to the Consumers. Download the Source

 

Copy from: https://simonwdixon.wordpress.com/2011/05/19/getting-started-with-rabbitmq-in-net-%E2%80%93-part-3/

 
Leave a comment

Posted by on November 18, 2015 in RabbitMQ

 

MSDTC on server … is unavailable

By Sochinda,

Please go to Run: services.msc

and find service’s called Distributed Transaction Coordinator and start

11

 

 
Leave a comment

Posted by on November 11, 2015 in C#, LinQ, SQL Server

 

Ubuntu Server Setup Guide for Django Websites

By: Brent O’Connor.

This guide is a walk-through on how to setup Ubuntu Server for hosting Django websites. The Django stack that will be used in this guide is Ubuntu, Nginx, Gunicorn and Postgres. This stack was chosen solely from the reading I’ve done and talking to other Django developers in order to get their recommendations. This stack seems to be one of the latest “standard” stacks for Django deployment. This guide also assumes that you’re familiar with Ubuntu server administration and Django. I needed an example site for this guide so I chose to use my Django Base Site which is available on Github.

I would also like to thank Ben Claar, Adam Fast, Jeff Triplett and Frank Wiles for their suggestions and input on this guide.

Step 1: Install Ubuntu Server

The version of Ubuntu I’m using for this guide is Ubuntu 11.10 64 bit Server. I’ve installed Ubuntu Server in a VirtualBox VM on my MacBook Pro which is currently running Mac OS X 10.7.2. During the installation of Ubuntu Server I answered the prompts with the following:

Language: English
Install Menu: Install Ubuntu Server
Select a language: English
Select your location: United States
Configure the Keyboard: No
Configure the keyboard: English (US)
Configure the keyboard: English (US)
Hostname: ubuntu-vm
Configure the clock: Yes
Partition disks: Guided - use entire disk and set up LVM
Partition disks: SCSI3 (0,0,0) (sda) - 21.5 GB ATA VBOX HARDDISK
Partition disks: Yes
Partition disks: Continue
Partition disks: Yes
Set up users and passwords: Brent O'Connor
Set up users and passwords: (Enter a username)
Set up users and passwords: ********
Set up users and passwords: ********
Set up users and passwords: No
Configure the package manager: <blank>
Configure taskse1: No automatic updates
Software selection: <Continue>
Install the GRUB boot loader on a hard disk: Yes
Installation complete: <Continue>

Step 2: Setup Port Forwarding

Under the settings for your VM in VirtualBox click on the “Network” tab and then click on the “Port Forwarding” button. Now click on the plus and add the following settings to setup port forwarding for web and ssh.

Name Protocol Host IP Host Port Guest IP Guest Port
SSH TCP 2222 22
Web TCP 8080 80

Step 3: Install Software

Before you begin it might be a good idea to update your system clock:

$ sudo ntpdate time.nist.gov

Download lists of new/upgradable packages:

$ sudo aptitude update

OpenSSH

Since I like to connect to my servers using SSH the first thing I install is openssh-server:

$ sudo aptitude install openssh-server

Since you setup port forwarding in step 2, you should now be able to open up your Terminal and connect to your Ubuntu Server using the following:

$ ssh localhost -p 2222

Python Header Files

The Python header files are needed in order to compile binding libraries like psycopg2.

$ sudo aptitude install python2.7-dev

PostgreSQL

$ sudo aptitude install postgresql postgresql-server-dev-9.1

Make your Ubuntu user a PostgreSQL superuser:

$ sudo su - postgres
$ createuser --superuser <your username>
$ exit

Restart PostgreSQL:

$ sudo /etc/init.d/postgresql restart

Nginx

$ sudo aptitude install nginx

Git

$ sudo aptitude install git

Step 4: Setup a Generic Deploy User

The reason we are setting up a generic deploy user is so that if you have multiple developers who are allowed to do deployments you can easily add the developer’s SSH public key to the deploy user’s/home/deploy/.ssh/authorized_keys file in order to allow them to do deployments.

$ sudo useradd -d /home/deploy -m -s /bin/bash deploy

Step 5: Install an Example Site

Setup a virtualenv:

$ sudo apt-get install python-setuptools
$ sudo easy_install pip virtualenv
$ cd /usr/local/
$ sudo mkdir virtualenvs
$ sudo chown deploy:deploy virtualenvs
$ sudo su deploy
$ cd virtualenvs
$ virtualenv --no-site-packages example-site
$ exit

Note

I personally use and setup virtualenvwrapper on all my servers and local development machines so that I can use workon <virtualenv> to easily activate a virtualenv. This is why I put all my virtualenvs in /usr/local/virtualenvs.

Make a location for the example site:

$ cd /srv/
$ sudo mkdir sites
$ sudo chown deploy:deploy sites
$ sudo su deploy
$ cd sites
$ git clone git://github.com/epicserve/django-base-site.git example-site
$ cd example-site/
$ git checkout -b example_site 5b05e2dbe5
$ echo `pwd` > /usr/local/virtualenvs/example-site/lib/python2.7/site-packages/django_project_root.pth
$ mkdir -p static/cache
$ exit
$ sudo chown www-data:www-data /srv/sites/example-site/static/cache
$ sudo su deploy

Create the file /srv/sites/example-site/config/settings/local.py and add the following. Make sure to change the password and then save the file. I usually use a random string generator to generate a new password for each new Postgresql database and user:

from base import *

LOCAL_SETTINGS_LOADED = True

DEBUG = True

INTERNAL_IPS = ('127.0.0.1', )

ADMINS = (
    ('Your Name', 'username@example.com'),
)

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': 'example_site',
        'USER': 'example_site',
        'PASSWORD': '<enter a new secure password>',
        'HOST': 'localhost',
    }
}

Install the sites required python packages:

$ source /usr/local/virtualenvs/example-site/bin/activate
$ cd /srv/sites/example-site/
$ pip install -r config/requirements/production.txt

Create a PostgreSQL user and database for your example-site:

# exit out of the deploy user account
$ exit
$ createuser example_site -P
$ Enter password for new role: [enter the same password you used in the local.py file from above]
$ Enter it again: [enter the password again]
$ Shall the new role be a superuser? (y/n) n
$ Shall the new role be allowed to create databases? (y/n) y
$ Shall the new role be allowed to create more new roles? (y/n) n
$ createdb example_site -O example_site

Step 6: Daemonize Gunicorn using Ubuntu’s Upstart

Create your Upstart configuration file:

$ sudo vi /etc/init/gunicorn_example-site.conf

Add the following and save the file:

description "upstart configuration for gunicorn example-site"

start on net-device-up
stop on shutdown

respawn

exec /usr/local/virtualenvs/example-site/bin/gunicorn_django -u www-data -c /srv/sites/example-site/config/gunicorn/example-site.py /srv/sites/example-site/config/settings/__init__.py

Start the gunicorn site:

$ sudo start gunicorn_example-site

Step 7: Setup Nginx to proxy to your new example site

Create a new file sudo vi /etc/nginx/sites-available/example-site.conf and add the following to the contents of the file:

server {

    listen       80;
    server_name  localhost;
    access_log   /var/log/nginx/example-site.access.log;
    error_log    /var/log/nginx/example-site.error.log;

    location = /biconcave {
        return  404;
    }

    location  /static/ {
        root  /srv/sites/example-site/;
    }

    location  /media/ {
        root  /srv/sites/example-site/;
    }


    location  / {
        proxy_pass            http://127.0.0.1:8000/;
        proxy_redirect        off;
        proxy_set_header      Host             $host;
        proxy_set_header      X-Real-IP        $remote_addr;
        proxy_set_header      X-Forwarded-For  $proxy_add_x_forwarded_for;
        client_max_body_size  10m;
    }

}

Enable the new site:

$ cd /etc/nginx/sites-enabled
$ sudo rm default
$ sudo ln -s ../sites-available/example-site.conf

Start nginx:

$ sudo /etc/init.d/nginx start

Step 8: Test the new example site

While still connected to your Ubuntu server via SSH run the following, which should spit out the HTML for your site:

wget -qO- 127.0.0.1:80

Since you setup port forwarding in step 2 for web, you should also be able to open up your browser on your local host machine and pull up the website using the URL, http://127.0.0.1:8080.

Copy from: http://epicserve-docs.readthedocs.org/en/latest/django/ubuntu-server-django-guide.html

 
Leave a comment

Posted by on November 4, 2015 in Django, Nginx, Python