Flvorful Bloggage: Tag part4

Deep in Rails: ActionMailer#deliver Part VI

jake — Sun, 20 Dec 2009 09:41:00 -0600

So now we know how Rails sends emails. What can we do with that knowledge?

All kinds of stuff.

One of my clients needed the ability to send messages through 3 different SMTP servers. They wanted it to be easy to use for their developers and allow them to change out SMTP servers for each mailer method. Now that we know how AM does it’s thing, we can use that knowledge to create a quick patch and solve the above problem.

We’ll create a new file in out Rails app inside lib/ and call it action_mailer_hacks.rb. Inside of the file add the following (dont worry, we’ll go over each line)

module ActionMailerHacks

    module InstanceMethods
        def deliver_with_switchable_smtp!(mail = @mail)
            unless logger.nil?
                logger.info  "Switching SMTP server to: #{new_smtp.inspect}" 
            end
            ActionMailer::Base.smtp_settings = new_smtp unless new_smtp.nil?
            deliver_without_switchable_smtp!(mail = @mail)
        end

    end

    def self.included(receiver)
        receiver.send :include, InstanceMethods
        receiver.class_eval do
            adv_attr_accessor :new_smtp
            alias_method_chain :deliver!, :switchable_smtp
        end
    end
end

ActionMailer::Base.send :include, ActionMailerHacks

First, in module InstanceMethods, we create a new deliver method that will switch the SMTP server from AM. We do some quick logging and then set AM::Base.smtp_settings to the new smtp server. After we do that, we call deliver_without_switchable_smtp which gets created when we do our alias_method_chaining later down the page.

When this module is included we include the InstanceMethods module into the receiver and then run class_eval so that we can create a new adv_attr_accessor called new_smtp and then call alias_method_chain to create our without method. After that we include this whole module into AM::Base and we’re done with the hack. Add require "action_mailer_hacks.rb" to your environment.rb file and we’re ready for the mailer.

Now we move on to MessageMailer and actually use the hack.

class MessageMailer < ActionMailer::Base
  def contact_form(msg_from, message)
    subject    'Im contacting you'
    recipients "someemail@somedomain.com" 
        new_smtp   :address => 'mail.someplace.com', :port => 25, :domain => 'someplace.com', :authentication => :login, :user_name => 'xxxxdddd', :password => 'secret'
    from       msg_from
    sent_on    Time.now
    body       :message => message
  end
end

All we do is call new_smtp with our new server address and then this method gets delivered through that SMTP server. That’s it. Simple and clean, exactly how I like it.

Well, that was fun. Next time, we’ll look through something a little shorter, maybe migrations. Until then, have fun and keep codin’

Deep in Rails: ActionMailer#deliver Part V

jake — Mon, 14 Dec 2009 12:54:00 -0600

So now we have our mail object. Let’s send it.

For the sending of the message, we need to go back to the beginning, method_missing:

(line 62 action_mailer/base.rb)
def method_missing(method_symbol, *parameters) #:nodoc:
  if match = matches_dynamic_method?(method_symbol)

    case match[1]
      when 'create'  then new(match[2], *parameters).mail
      when 'deliver' then new(match[2], *parameters).deliver!
      when 'new'     then nil
      else super
    end
  else
    super
  end
end

You remember method_missing from a long time ago. We just finished the first part of the line:

when 'deliver' then new(match[2], *parameters).deliver!

We created the new AM object and now we call deliver! on it. That’s our next stop:

(line 520 action_mailer/base.rb)
def deliver!(mail = @mail)
  raise "no mail object available for delivery!" unless mail
  unless logger.nil?
    logger.info  "Sent mail to #{Array(recipients).join(', ')}"
    logger.debug "\n#{mail.encoded}"
  end

  begin
    __send__("perform_delivery_#{delivery_method}", mail) if perform_deliveries
  rescue Exception => e  # Net::SMTP errors or sendmail pipe errors
    raise e if raise_delivery_errors
  end

  return mail
end

The beginning of this method does some checking and logging. The main part of this method is:

__send__("perform_delivery_#{delivery_method}", mail) if perform_deliveries

We are again using __send__ to bypass any possible overridden method and calling perform_delivery_#{delivery_method} with the mail object. delivery_method is a configuration attribute on the AM class along with perform_deliveries. AM defaults to “smtp” for delivery_method and “true” for perform_deliveries so the method we end up calling in our example is perform_delivery_smtp. Let’s see that code:

(line 680 action_mailer/base.rb)
def perform_delivery_smtp(mail)
  destinations = mail.destinations
  mail.ready_to_send
  sender = (mail['return-path'] && mail['return-path'].spec) || mail.from

  smtp = Net::SMTP.new(smtp_settings[:address], smtp_settings[:port])
  smtp.enable_starttls_auto if smtp_settings[:enable_starttls_auto] && smtp.respond_to?(:enable_starttls_auto)
  smtp.start(smtp_settings[:domain], smtp_settings[:user_name], smtp_settings[:password],
             smtp_settings[:authentication]) do |smtp|
    smtp.sendmail(mail.encoded, sender, destinations)
  end
end

This is the finale. The climax. The whole reason we even engage in ActionMailer, to actualy send an email. Here we go.

The first line pulls the destinations (recipients) from the mail object and stores them in a variable. Next, we call ready_to_send on the mail object which is also the TMail object (remember that we assigned the final TMail object to the @mail attribute on our AM object). This method is defined in TMail::Net:

(line 62 action_mailer/tmail-1.2.3/tmail-1.2.3/net.rb)
def ready_to_send
  delete_no_send_fields
  add_message_id
  add_date
end

NOSEND_FIELDS = %w(
  received
  bcc
)

def delete_no_send_fields
  NOSEND_FIELDS.each do |nm|
    delete nm
  end
  delete_if {|n,v| v.empty? }
end

def add_message_id( fqdn = nil )
  self.message_id = ::TMail::new_message_id(fqdn)
end

def add_date
  self.date = Time.now
end

Now, if you scan through that code you may notice that we are calling delete and delete_if like we were in a standard Enumerable object, but we are not. TMail also defines those methods:

def delete( key )
  @header.delete key.downcase
end

def delete_if
  @header.delete_if do |key,val|
    if Array === val
      val.delete_if {|v| yield key, v }
      val.empty?
    else
      yield key, val
    end
  end
end

What we are doing with all this is stripping and setting some defaults. delete_no_send_fields get’s rid of the “received” and “bcc” keys from our header and we are getting rid of any headers that have empty values. After that we set a nice new hex type id for this email with TMail.new_message_id which is defined in TMail::Utils which gives us a string with a hex code attached to your machine’s hostname. The last line adds the date to the mail message and that’s it.

The next line sets the sender attribute.

sender = (mail['return-path'] && mail['return-path'].spec) || mail.from

Now we get into the actual sending of the email. We aren’t going to go into it because it is part of Ruby’s core lib, but we will quickly skim over it to see what’s going on:

smtp = Net::SMTP.new(smtp_settings[:address], smtp_settings[:port])
smtp.enable_starttls_auto if smtp_settings[:enable_starttls_auto] && smtp.respond_to?(:enable_starttls_auto)
smtp.start(smtp_settings[:domain], smtp_settings[:user_name], smtp_settings[:password],
           smtp_settings[:authentication]) do |smtp|
  smtp.sendmail(mail.encoded, sender, destinations)
end

First, we create the new SMTP object and then we enable StartTLS if necessary (this allows you to send email through SMTP hosts that require SSL connections). Next we login to the SMTP server (with the start command) and call sendmail inside the block to do the actual delivery.

We’re done. That’s how an email gets sent in Rails. Next week we will create a quick monkey patch for ActionMailer based on the information that we learned from the trek.

Deep in Rails: ActionMailer#deliver Part IV

jake — Sun, 06 Dec 2009 09:42:00 -0600

So we just opened up CompiledTemplates and defined (or compiled) our unique method for this particular template. Now we continue with render:

# (line 193 action_pack/action_view/renderable.rb)
def render(view, local_assigns = {})
  compile(local_assigns)

  view.with_template self do
    view.send(:_evaluate_assigns_and_ivars)
    view.send(:_set_controller_content_type, mime_type) if respond_to?(:mime_type)

    view.send(method_name(local_assigns), local_assigns) do |*names|
      ivar = :@_proc_for_layout
      if !view.instance_variable_defined?(:"@content_for_#{names.first}") && view.instance_variable_defined?(ivar) && (proc = view.instance_variable_get(ivar))
        view.capture(*names, &proc)
      elsif view.instance_variable_defined?(ivar = :"@content_for_#{names.first || :layout}")
        view.instance_variable_get(ivar)
      end
    end
  end
end

and our params:

view => #"Send this message"}, 
          @helpers=#, 
          @controller=#"Send this message"}, 
              @mailer_name="message_mailer", 
              @recipients="someemail@somedomain.com", 
              @parts=[], 
              @template=#, 
              @charset="utf-8">, 
          @_current_render=nil, 
          @assigns_added=nil, 
          @_first_render=nil, 
          @view_paths=["/example_app/app/views"]>
local_assigns => {}

The first thing we do after we compile our unique method is call with_template on view which is a convenient method that allows us to temporarily swap out the current template for the view, set it to another one, do some processing and then return the original template back to the view. Let’s see with_template:

# (line 298 action_pack/action_view/base.rb)
def with_template(current_template)
  last_template, self.template = template, current_template
  yield
ensure
  self.template = last_template
end

Quite simple indeed. Now let’s see exactly what we are doing to the template. The first thing we do is send the method _evaluate_assigns_and_ivars to the view. This is private method in ActionView::Base which creates instance variables for each key in the `assigns@ attribute hash. This hash is filled with ourbodycall in the mailer model you create. Here isevaluateassignsandivars`:

# (line 307 action_pack/action_view/base.rb)
def _evaluate_assigns_and_ivars #:nodoc:
 unless @assigns_added
   @assigns.each { |key, value| instance_variable_set("@#{key}", value) }
   _copy_ivars_from_controller
   @assigns_added = true
 end
end

After the instance variables are set, we call _copy_ivars_from_controller which does exactly that, copies the instance variables from this view’s controller to this view. This is how instance variables that you create in normal controller methods are accessed in the view. Let’s look at that definition:

# (line 315 action_pack/action_view/base.rb)
def _copy_ivars_from_controller #:nodoc:
  if @controller
    variables = @controller.instance_variable_names
    variables -= @controller.protected_instance_variables if @controller.respond_to?(:protected_instance_variables)
    variables.each { |name| instance_variable_set(name, @controller.instance_variable_get(name)) }
  end
end

This method finds all proper instance variables in the attached controller and sets the same instance variables in the current view object. Once we have copied the instance variables we move to the next line in render:

view.send(:_set_controller_content_type, mime_type) if respond_to?(:mime_type)

This is another private method in ActionView::Base and it’s job is quite simple: set the content_type for the response attribute if this view’s controller has one. Let’s look at the code:

# (line 323 action_pack/action_view/base.rb)
def _set_controller_content_type(content_type) #:nodoc:
  if controller.respond_to?(:response)
    controller.response.content_type ||= content_type
  end
end

In our case, the controller is the MessageMailer object and it doesnt respond to response so we dont set a content_type. Now we come back to render:

# (line 193 action_pack/action_view/renderable.rb)
def render(view, local_assigns = {})
  ...
  view.with_template self do
    ...
    view.send(method_name(local_assigns), local_assigns) do |*names|
      ivar = :@_proc_for_layout
      if !view.instance_variable_defined?(:"@content_for_#{names.first}") && view.instance_variable_defined?(ivar) && (proc = view.instance_variable_get(ivar))
        view.capture(*names, &proc)
      elsif view.instance_variable_defined?(ivar = :"@content_for_#{names.first || :layout}")
        view.instance_variable_get(ivar)
      end
    end
  end
end

The next thing we are going to do is call our dynamically generated (and now compiled) view method for this template. We call it with a block that is used to capture content pieces that this view may use (think content_for). AM does not use any of these so we end up skipping over this whole section and just executing our template method. In a future series on ActionController, we will dive more into what the block does and how Rails uses it’s information, but for now we will move on.

So what exactly did we do when we called that dynamically generated method? Let’s find out. First, we will look at our method again:

def _run_erb_app47views47message_mailer47contact_form46erb(local_assigns)
        old_output_buffer = output_buffer;;@output_buffer = '';  __in_erb_template=true ; @output_buffer.concat "Hi,\n"
@output_buffer.concat "\n"
@output_buffer.concat "I am contacting you with this message:\n"
@output_buffer.concat "\n"
@output_buffer.concat(( @message ).to_s); @output_buffer.concat "\n"
@output_buffer.concat "\n"
@output_buffer.concat "\n"
@output_buffer.concat "\n"@output_buffer.concat "Thanks\n"
@output_buffer.concat "\n"
@output_buffer.concat "Jake"; @output_buffer
ensure
  self.output_buffer = old_output_buffer
end

Remember that output_buffer is an attribute inside of ActionView::Base and that it holds the current output for the view that is being processed. Our dynamic method first stores the current output_buffer and then resets it so that it can insert the newly parsed message. When we execute our method, we concat our entire view (with any ruby processing) into the output_buffer. So after we execute _run_erb_app47views47message_mailer47contact_form46erb our output_buffer buffer looks like:

@output_buffer => "Hi,\n\nI am contacting you with this message:\n\nSend this message\n\n\n\nThanks\n\nJake"

which is the final email body that will be sent.

Alright, now we have rendered our message and we hit the bottom of this method chain. We now pop all the way back up create!. I know, I know. It was so long ago that you dont remember. Here is a refresher:

(line 458 action_mailer/base.rb)
def create!(method_name, *parameters) #:nodoc:
  unless String === @body
    ...
    @body = render_message(@template, @body) if template_exists

    if !@parts.empty? && String === @body
      @parts.unshift Part.new(:charset => charset, :body => @body)
      @body = nil
    end
  end

  @mime_version ||= "1.0" if !@parts.empty?

  @mail = create_mail
end

We just got through with render_message and now we have our @body attribute filled with our message. The next line checks to see if we have parts and if we have a body that’s a String. If we do, we create a new Part with the current @body and add it to all of our parts. After that, we get rid of @body. We do this so that create_mail doesnt try and rerender the body again if we have other parts. Since we dont, we skip over this and go straight to:

@mail = create_mail

Now we’re gettin’ somewhere. This starts the actual email processing part of AM. Let’s see create_mail:

(line 638 action_mailer/base.rb)
def create_mail
  m = TMail::Mail.new

  m.subject,     = quote_any_if_necessary(charset, subject)
  m.to, m.from   = quote_any_address_if_necessary(charset, recipients, from)
  m.bcc          = quote_address_if_necessary(bcc, charset) unless bcc.nil?
  m.cc           = quote_address_if_necessary(cc, charset) unless cc.nil?
  m.reply_to     = quote_address_if_necessary(reply_to, charset) unless reply_to.nil?
  m.mime_version = mime_version unless mime_version.nil?
  m.date         = sent_on.to_time rescue sent_on if sent_on

  headers.each { |k, v| m[k] = v }

  real_content_type, ctype_attrs = parse_content_type

  if @parts.empty?
    m.set_content_type(real_content_type, nil, ctype_attrs)
    m.body = normalize_new_lines(body)
  else
    if String === body
      part = TMail::Mail.new
      part.body = normalize_new_lines(body)
      part.set_content_type(real_content_type, nil, ctype_attrs)
      part.set_content_disposition "inline"
      m.parts << part
    end

    @parts.each do |p|
      part = (TMail::Mail === p ? p : p.to_mail(self))
      m.parts << part
    end

    if real_content_type =~ /multipart/
      ctype_attrs.delete "charset"
      m.set_content_type(real_content_type, nil, ctype_attrs)
    end
  end

  @mail = m
end

The first thing we do is initialize a new TMail::Mail instance and then we go on to assign several attributes for it.

m.subject,     = quote_any_if_necessary(charset, subject)
m.to, m.from   = quote_any_address_if_necessary(charset, recipients, from)
m.bcc          = quote_address_if_necessary(bcc, charset) unless bcc.nil?
m.cc           = quote_address_if_necessary(cc, charset) unless cc.nil?
m.reply_to     = quote_address_if_necessary(reply_to, charset) unless reply_to.nil?
m.mime_version = mime_version unless mime_version.nil?
m.date         = sent_on.to_time rescue sent_on if sent_on

The quote_any_if_necessary, quote_any_address_if_necessary and quote_address_if_necessary are helper methods included from the module ActionMailer::Quoting. Let’s take a gander at those methods:

(line 5 action_mailer/quoting.rb)

def quoted_printable(text, charset)
  text = text.gsub( /[^a-z ]/i ) { quoted_printable_encode($&) }.gsub( / /, "_" )
  "=?#{charset}?Q?#{text}?="
end

def quoted_printable_encode(character)
  result = ""
  character.each_byte { |b| result << "=%02X" % b }
  result
end

if !defined?(CHARS_NEEDING_QUOTING)
  CHARS_NEEDING_QUOTING = /[\000-\011\013\014\016-\037\177-\377]/
end

def quote_if_necessary(text, charset)
  text = text.dup.force_encoding(Encoding::ASCII_8BIT) if text.respond_to?(:force_encoding)

  (text =~ CHARS_NEEDING_QUOTING) ?
    quoted_printable(text, charset) :
    text
end

def quote_any_if_necessary(charset, *args)
  args.map { |v| quote_if_necessary(v, charset) }
end

def quote_address_if_necessary(address, charset)
  if Array === address
    address.map { |a| quote_address_if_necessary(a, charset) }
  elsif address =~ /^(\S.*)\s+(<.*>)$/
    address = $2
    phrase = quote_if_necessary($1.gsub(/^['"](.*)['"]$/, '\1'), charset)
    "\"#{phrase}\" #{address}"
  else
    address
  end
end

def quote_any_address_if_necessary(charset, *args)
  args.map { |v| quote_address_if_necessary(v, charset) }
end

Im not going to bore you with the inane details, needless to say, these methods take illegal characters and quote them if necessary. Once these are quoted and sanitized, they are inserted into the Mail object. After we initialize those attributes, we set the header attributes for the Mail object with the following line:

headers.each { |k, v| m[k] = v }

real_content_type, ctype_attrs = parse_content_type

We cycle through all the headers of our mailer object and stick them into the TMail::Mail object. Next we call parse_content_type which is a private method that we get from ActionMailer::PartContainer:

(line 43 action_mailer/part_container.rb)
def parse_content_type(defaults=nil)
  if content_type.blank? 
    return defaults ? 
      [ defaults.content_type, { 'charset' => defaults.charset } ] : 
      [ nil, {} ] 
  end 
  ctype, *attrs = content_type.split(/;\s*/)
  attrs = attrs.inject({}) { |h,s| k,v = s.split(/=/, 2); h[k] = v; h }
  [ctype, {"charset" => charset || defaults && defaults.charset}.merge(attrs)]
end

All we are doing here is parsing the contenttype of the mailer (in our case “text/plain”) and returning an array with the contenttype and a hash. Let’s see what our example ends up returning:

parse_content_type => ["text/plain", {"charset"=>"utf-8"}]
real_content_type => "text/plain"
ctype_attrs => {"charset"=>"utf-8"}

Now we move on to:

if @parts.empty?
  m.set_content_type(real_content_type, nil, ctype_attrs)
  m.body = normalize_new_lines(body)
else
  ...
end

In our case, @parts is empty so we set the content type of our TMail object with the results from our parsing. After that, we set the body of the TMail object to the current AM object’s body, but we first do some sanitation duties with normalize_new_lines which is a method in ActionMailer::Utils. In fact, it is the only method in that module:

(action_mailer/utils.rb)
module ActionMailer
  module Utils #:nodoc:
    def normalize_new_lines(text)
      text.to_s.gsub(/\r\n?/, "\n")
    end
  end
end

What we are doing here is cleansing Microsoft evil from the message. Windows likes to use “\r\n” for line breaks in text, the rest of computing thinks it’s enough to just use “\n”. I agree with the rest of computing. Anyway, this method changes all “\r\n“‘s to “\n“‘s so that our email message parses correctly on all email servers.

After we normalize, we are done with setting up the object and the final line of create_mail is:

@mail = m

Now we set the AM attribute @mail to the newly instantiated TMail object and we’re done… with the creation of the email message. Now we have to actually send it. We’ll get to that next. See ya then.