Ask them to brief explain their methodology and most important the KPI's that they will track thru your project.
That way you can compare different agencies KPI's and see the ones that make sense with your business goals.

A few ones to exemplify:
General: Traffic, Sources, Conversion Rate, Bounce Rate.
Adwords: ROI, CPA, CPC e CTR.
SEO: Inbound Keywords, Rankings, External Links (Volume and Quality).
Social: Audience, Reach, Engagment.

and a lot more...

Hope that helps.
If you need more info feel free to give me a call.

I've been on both sides of the table in this conversation a number of times and have come to the conclusion that the only factor that matters is sales generation.

Page ranking, etc. is all meaningless if their efforts don't result in increased sales - significantly in excess of their fees and your advertising investment.

Don't forget that driving traffic is only half of the equation. Conversion is the other - and perhaps the more important / difficult - part of the equation.

Don't pay someone just to drive traffic without a strategy to convert visitors into customers. Otherwise you're wasting your time and your money.

Ask them for specific sales results they have been able to generate for other clients. Compare the sales increases to the costs of their fees and advertising expenses. Ask for and call references of current / past clients they have generated results for.

I'm not quite sure what you mean by "an ecommerce site to compare it to another one" but, in addition to traffic stats, be sure to look at conversion percentages and sales numbers.

To your success, Frank

1. Before you talk to them make sure you are clear about your goals and strategy.
Outsourcing any part of your business or hiring a firm to do work for you should be like hiring an employee. You should have a VERY clear idea about what you want them to accomplish. After all - you're hiring them to help you accomplish your goals... not defaulting to their strategy to accomplish theirs (remember - they are a business too).

2. Before you talk to them be certain you know what your reasonable Customer Acquisition Cost (CAC) is. This is the number you'll use to know if you can afford the services. When discussing pricing - don't mention this to them - let them TELL you the costs of their services. But knowing this will allow you to decide if you can afford the investment.

3. Find out what they do to mitigate risk for their customers. In other words - what are they willing to offer YOU to ensure YOUR success. If they want a lump sum payment that's non-refundable... run for the hills (unless they offer some other amazing guarantees, etc.).

For assistance figuring any of the above out - or for any other help - give me a call.

Best of luck!!

For SEO, ask the following questions mentioned by Rand Fishkin (he really knows his stuff)
http://moz.com/blog/ten-question-litmus-test-for-professional-seos

For Adwords, use these questions from Wordstream since they know Paid Search inside out!
http://www.wordstream.com/blog/ws/2011/06/22/ppc-agency-questions

For Social Media, David really knows his stuff about that. He created a list of questions you should ask. http://www.therisetothetop.com/davids-blog/key-questions-hire-social-media-marketing-consultant/

Hope this helps!

On the SEM side, we created a list of questions we recommend our clients ask their potential vendors. You can access it at:
http://www.globalsempartners.com/ppc-agency-questions.docx

Let me if you have any questions, I'd be happy to jump on a call.

SEO is all about content, and because the content must generate a sense of urgency among prospects if it is to be effective. Content should challenge them to question the effectiveness of their current methods and encourage them to explore other more efficient and effective alternatives. Ask your mkt agency about how well they can deal with content marketing.
Besides if you do have any questions give me a call: https://clarity.fm/joy-brotonath