App Engine's Email service future

From time to time I have seen posts in app engine's groups of people complaining for Email service glitches. Some of those real some not so. Sometimes coming from people who did not bother to read the documentation or just ignorant of what an Email service is all about.
App Engines team response alarmed me a little.
IMHO it is not a good policy for GAE to abandon(?) services middle way
instead of improving - enhancing those.
so I wrote the following post and now I am waiting to see how the conversation evolves.

@Icai
I agree on most of what you write above, and I understand that you prefer to focus on more important things, also having run Email services for enterprises in the past I do know it is not trivial.
But ....
still I believe Email service is a major asset for GAE and dropping it (or anything to that effect) will constitute a major blow to App Engine.
Gae offers a limited subset of services compared to what a LAMP box or a IaaS box can offer but been a PaaS provides trouble free operation and automatic scalability.
Email service usually is part of any web operation so by dropping it out you farther limit the number of potential applications that fit well into what GAE offers.
Of course, developers can look into alternative options but this makes our life difficult since we have to integrate several other third party services in order to make a working web solution i.e. setting up multiple accounts, feed traffic back and forth to other services, having to monitor and deal with one more possible point of failure. All this defies to some extend the benefits of GAE as a PaaS.
Also, dropping a service in a time when competition is adding services, will sent the wrong signal to App Engine's developers/users echo system and having in mind that G is associated with the best email service can possibly turn into a PR disaster.

Further more, been a regular reader of the groups and following App Engine since the very early days I do not see that Email service has raised a lot of issues. I believe for most people who know what they are doing and do not abuse the service it works quite smoothly. Some of the issues raised (mainly spam flagging)
a) happen to the best of Email services b) are addressed by well known techniques and practices described by others here and elsewhere.
In conclusion:
I would welcome any measure taken to fight service abuse like using GAE primarily as a mail server - we all understand that this is not what GAE is all about.
Instead of dropping the service I would prefer to consider:
a) put false positive spam flagging issues under the responsibility of developers.
b) exclude the service or part of it (like delivery assurances) from the future SLA offer.
c) think about the technical possibility to integrate it to gmail which is the *most* reliable email service in town.

Selecting distinct entities across a large table

I have faced this kind of problem some time ago.
I tried some of the solutions suggested below (in memory sort and filtering, encoding things into keys etc. and I have benchmarked those for both latency and cpu cycles using some test data around 100K entities)
An other approach I have taken is encoding the date as an integer (day since start of epoch or day since start of year, same for hour of day or month depending on how much detail you need in your output) and saving this into a property. This way you turn your date query filter into an equality only filter which does not even needs to specify an index) then you can sort or filter on other properties.
Benchmarking the latest solution I have found that when the filtered result set is a small fraction of the unfiltered original set, is 1+ order of magnitude faster and cpu-eficient. Worst case when no reduction of the result set due to filtering the latency and cpu usage was comparable to the previous solutions)

Hope this helps, or did I missed something ?
Happy coding-:)

Re: Policy for instance startup

Not been a Googler can't help much with this.
Having said that, I suspect there is a kind of build in algorithm that
does some kind of application profiling taking into acount QPS,
response times, and other parameters which adjusts instance life time,
number of instances to start etc..
This could possibly explain the difference in behaviour between your
staging and production appls.

happy coding;-)

World Countries and IP geocoding API for App Engine

I have been writing here again and again about the inherent problem that App Engine based Applications have in using third party API’s with quota limits based on ip addresses since all of those are served from the same block of ips alocated by Google  © and therefore have to share those quotas with other applications hosted on App Engine using the same service.

Today I am happy to offer a free IP geocoding and world countries information service API and hope this solves to some extend the problem of Server side IP geocoding for fellow App Engine developers.

Detailed service description and more information is provided here : "World countries API"

Happy coding:-)

maximum number of requests that may be handled in a single process lifetime

I was doing some load tests on app engine today when I noticed a new Info message in the logs: "After handling this request, the process that handled this request reached the maximum number of requests that may be handled in a single process' lifetime, and exited normally."

So what that supposed to mean ?
Up to know we new that application instances are automatically terminated after some inactivity time out. If I understand this message well now we know that a process can be terminated after handling so many requests. How many exactly ? is this a new magic number ? Lets hope we will have some definite answers from the always helpful App Engines team.

I am doing some latency optimization (lazy imports) in this app based on the assumption that an instance will stay live as long as there are more requests coming in and I was scared this new number spoils my optimization logic. After some more testing I have found out that this is not true since whatever this number is it must be in the order of thousands requests - so no big trouble in my use case. Still I think App Engine's developers deserve to know more about such parameters, it helps both developers and the platform.

Downloan app code feature or bug ? - download_app

I think App Engines developers community has by a vast majority rejected the idea of code downloading at least as default (opt out) option.
I also do not like the idea of a payable service, since it :

• will complicate the pricing model
• will attract criticism against the platform and 
• help guys who are in the business of doing unreasonable GAE vs S3 vs whatever_looks_like_cloud comparisons happy.

But ... then again who am I to tell mother G what to do ? -:)
( see my post at google-appengine group )

Instances console in App engine's Admin Tools

It seems a new feature has just been rolled out in App Engine's production Admin tools - instances - it shows a view of total running instances along with QPS, Latency and memory used Memory as well as averages for the above values.
There is also a summary of the above in application's dashboard.
It is a very useful feature that will help a lot with application performance monitoring and resolving issues on when and how many new instances are started and killed.
Well done dev team. :-)