eduardofv blog posts

Ley de Benford para Municipios de México

Contribuí un dataset al proyecto de Testing Benford's Law con los datos de la población de los municipios de México según el Censo 2010 de INEGI

VisualizaciÃƒÂ³n de Densidad Poblacional de MÃƒÂ©xico

Gracias a los datos recién publicados por INEGI del Censo 2010 de Población y Vivienda estoy haciendo algunas visualizaciones en Processing.

Usando el método descrito en dos posts anteriores generé la siguiente imagen tipo heatmap o mapa de calor para la densidad poblacional de todo México. Básicamente lo que hice fue:

Tomar de aquí los datos de todas las poblaciones en formato de texto tal como los provee INEGI.
Generar un archivo tipo CSV con las coordenadas geográficas de las pobaciones con al menos 100 habitantes y el logaritmo base 10 de esa cantidad para facililitar el manejo. Son 52,856 ubicaciones. El archivo está disponible aquí
En Processing se grafica cada ubicación con un círculo cuyo color, radio y transparencia es proporcional al tamaño de la población del lugar. Cada círculo tiene un pequeño degradado para conseguir un efecto más suave en la imagen.

El resultado es la siguiente imagen (click para una imagen de mayor resolución):

El objetivo de la visualización no es la precisión geográfica sino la sensación de la densidad poblacional. Las ciudades más grandes son fácilmente identificables, así como las zonas metropolitanas. Hay grandes areas poco habitadas de las que podemos imaginar sus características geográficas.

Links a imágenes a diferentes resoluciones y al proyecto en Github:

Los datos son información pública de INEGI y no alterados mas que en formato para poder ser procesados. Las imágenes tienen licencia Creative Commons Attribution 3.0 Unported (These files are licensed under the Creative Commons Attribution 3.0 Unported license. You are free: to share Ã¯Â¿Â½ to copy, distribute and transmit the work to remix Ã¯Â¿Â½ to adapt the work Under the following conditions: attribution Ã¯Â¿Â½ You must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work). El cÃ¯Â¿Â½digo es OpenSource bajo MIT License

The Freezer

Update: Freezer esta descontinuado

I've added a new function to this blog: The Freezer. It's a Web Cache (or Internet Archive) function that keeps a local copy of some pages that are referenced by the blog posts, so you can always go back to the exact version of the page when it was originally referenced. Web Pages changes or dies continously, and it's annoying when some useful information is lost when you need it.

The Freezer downloads the desired page, deletes any javascript it founds, tries to download every image and CSS files (and images referenced on the CSS) and modifies the links to point to the local copies. The purpose is to keep a snapshot of the page, obvioulsy not a working copy of it.

On the Blog Post, the links to freezed pages (icicles) are modified to point to The Freezer, and a external-link icon is added to reference to the actual page. Also, the content of the freezed pages is indexed so the Search function of the site can find inside it's content or as part of the original post.

The Freezer can be accesed whenever a frozen content is accesed or using the link on the right bar, which shows a list of the archived content that's available.

If you think any of the freezed content should not be cached for any reason, drop me a message and I'll remove it as soon as possible.

Otros Mapas Georeferenciados de MÃƒÂ©xico para VisualizaciÃƒÂ³n

Continuando con el tema del post anterior sobre la creación de visualizaciones de mapas georeferenciados de México he hecho un par de mapas más que pueden dar buenos resultados.

Todos están basados en este Mapa que tiene bien marcados los límites visibles en coordenadas geográficas y que es una proyección equirectangular. Yo eliminé la información hidrográfica e hice varias versiones incluyendo o no los estados y con diferentes colores. Los límites son los siguientes:

float lng_max=-86.3;
float lng_min=-118.5;
float lat_max=33.3;
float lat_min=14.2;

SVG con estados, colores originales
SVG con estados, sin colores
SVG sin estados, sin colores
SVG sin estados, en azul-blanco-negro. Este me sirvió para diferenciar el fondo de líneas principales y de detalles que excluí de mapas posteriores
SVG en fondo negro y lineas blancas para algunas visualizaciones
PNG sin estados, fondo blanco y líneas negras
PNG sin estados, fondo negro y líneas grises, ténues
PNG sin estados, fondo negro y líneas blancas

Ejemplo:

Licencia: Creative Commons Attribution 3.0 Unported

These files are licensed under the Creative Commons Attribution 3.0 Unported license. You are free: to share – to copy, distribute and transmit the work to remix – to adapt the work Under the following conditions: attribution – You must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work).

CÃƒÂ³mo Hacer Mapas Georeferenciados de MÃƒÂ©xico para VisualizaciÃƒÂ³n

La visualización de información es uno de los aspectos fundamentales del análisis de datos en la actualidad, como en el caso de la reciente visualización de amistad en Facebook que tuvo gran éxito

En OCCMundial estamos trabajando en geolocalización de ofertas de trabajo. Obtuvimos datos muy interesantes pero al tratar de visualizarlos con un mapa de fondo para hacer más interesante la presentación de los datos me encontré con el problema de conseguir un mapa georeferenciado sobre el cual mostrar la información. De otras regiones tales como el mundo completo, Estados Unidos o Europa es sencillo conseguirlos pero fue más complicado en el caso de México. Tampoco encontré un procedimiento sencillo para generarlo, por lo que en este post pongo mi experiencia y los resultados esperando que le sirvan a alguien más. La idea vino de Minigis que lo usa en un mapa mundial.

El problema consiste en conseguir una imagen de un mapa en el cual se tenga una correspondencia entre coordenadas geográficas y los pixeles. Los buenos mapas que escalan con el nivel de zoom son imágenes vectoriales.Wikimedia es un excelente recurso para comenzar. Los mapas estan en formato SVG. Aquí se encuentran los mapas de México. Es importante fijarse en el tipo de proyección ya que afecta el aspecto del mapa. Para este caso necesitamos un mapa equirectangular en el cual las coordenadas geográficas se pueden mapear de forma lineal con los pixeles de la imagen. Aunque el mejor mapa de México que conseguí tiene división política por municipios, lo que lo hace pesado y no muy bonito para algunos casos, pero es suficientemente bueno para nuestro uso.

Desafortunadamente el mapa no especifica las coordenadas que comprende por lo que es necesario averiguarlas. Usando Inkscape abrí el mapa y encontré que tiene marcados paralelos y meridianos cada 5 grados. Dado que el primer meridiano que se muestra es el 15 y está en la posición Y=48.4, el siguiente es el 20 y está en la posición Y=173.739 podemos comprobar que cada 5 grados corresponden a 125 pixeles, 1 grado a 2.06 pixeles y 0.03999 grados a 1 pixel. De la misma forma podemos comprobar para los paralelos ya que el último paralelo mostrado es el 115W (-115.0) en posición X=56.79 y se mantiene la misma proporción de grados a pixeles (lo que es correcto dado que es una proyección equirectangular). Con esto podemos calcular que el origen de la imagen se encuentra en 13.064° de latitud y -117.271° de longitud (dicho sea de paso, usaremos grados decimales por simplicidad).

Con esto se puede generar una imagen bitmap para ser usado en algún programa o con lenguaje de programación. En este caso estoy usando Processing para generar las visualizaciones. Importante, en general en las imágenes raster el origen no se encuentra en la esquina inferior izquierda como en el SVG sino en la superior izquierda, por lo que el origen se encuentra en 34.6586° lat y -117.271° long, que son las coordenadas se deben usar para calcular las posiciones de los pixeles. La longitud mínima se encuentra en -86.54. Para calcular los pixeles se usan las funciones lineales descritas abajo. La derivación es sencilla por lo que dejo solo una imagen de mis notas por si le sirve a alguien.

En el sketch de processing se puede ver una prueba con algunos puntos interesantes para comprobar que funciona. IMPORTANTE: los datos son muy aproximados, por lo que no recomendaría usar estos métodos para algo mas que visualizaciones sencillas, no mediciones o para ubicar exactamente alguna posición.

Cómo usar:

Se puede usar esta imagen con las dimensiones descritas arriba y las siguientes funciones para calcular pixeles a partir de coordenadas.

 
  int x = int( (117.271-lng)*770/(117.271-86.54) );
  int y = int( (34.6586-lat)*540/(34.6586-13.064) );

Otra imagen, con fondo negro que suele ser mejor para visualizaciones.

En caso de requerir una imagen de otro tamaño no es dificil escalar estas imágenes y encontrar las correspondientes funciones.

Sketch de Processing bajar

//Mantiene las mismas dimensiones del SVG original
int imgx=770;
int imgy=540;

void setup()
{
  //También se puede bajar con la URL directa a Wikimedia 
  //http://commons.wikimedia.org/wiki/File:Municipalities_of_Mexico_(equirectangular_projection).svg
  PShape mx = loadShape( "Municipalities_of_Mexico_(equirectangular_projection).svg" );
 
  size(imgx,imgy);
  shape(mx,0,0,imgx,imgy);
  smooth();
  noLoop();
  
  //Puntos de prueba
  plotAt(19.43333,-99.1333333,5); //DF
  plotAt(25.6666,-100.5, 5);   //MTY
  plotAt(20.6666,-103.3333, 5);  //GDL
 
  plotAt(32.534, -117.123,5); //Tijuana 
  plotAt(21.227, -86.7199,5); //Isla Mujeres
  
  plotAt(14.5176, -92.2199, 5); //Sur
  plotAt(17.8212, -89.1551, 5); //Belice
  plotAt(20.7959, -90.4006, 5); //Campeche
  plotAt(25.9393, -97.1231, 5); //Tamps
  plotAt(29.8605, -102.2996, 5); //Coah Nte
  plotAt(31.7820, -106.5058, 5); //Chih
  plotAt(32.7125, -114.7015, 5); //Nte
  plotAt(22.8865, -109.9164, 5); //Cabos
  plotAt(20.6,-100.38333, 5); //Qro
  plotAt(16.85,-99.93333, 5); //Acapulco
  
  save("mexico.jpg");
}

void plotAt(float lat, float lng, float rad)
{
  lng=-lng; //Si viene en long oeste
  int x = int( (117.271-lng)*770/(117.271-86.54) );
  int y = int( (34.6586-lat)*540/(34.6586-13.064) );
  

  println(x);
  println(y);
  
  fill(10,10,60,80);
  ellipse(x,y,rad,rad);
 }

Espejo del mapa SVG

Imagen con los puntos de prueba

Visualización - Algunas ofertas de empleo actuales (naranja), y principales ciudades (azul)

Arduino RGB LED HSV "Color Wheel"

Recently I've been playing with Arduino, Scratch and the 3pi robot for a personal project that I hope will be interesting enough to show soon. (I've got a lot of motivation from these guys specially the little one with big eyes).

To start I made this little example of a RGB Led. Usually you begin making a loop through the RGB scale of colors. But it doesn't look very natural since you expect to loop through a color hue. What you need is to convert a HSV (Hue Saturation Value) scale to RGB (Red Green Blue) scale that the LED support, and then loop through the Hue domain mantaining the Saturation and Value constants.

Circuit is here, look for the RGB LED example

Video:

Code (I've coded it as close as I could to the Arduino sample guidelines):


/*
 * Color Wheel LED
 *
 * Loops a RGB LED attached to pins 9,10,11 through
 * all the "exterior" colors of a Color Wheel
 * 
 * The RGB LED uses a 3 component (red green blue) model which adds light colors to produce 
 * a composite color. But the RGB does not make easy to loop through a more
 * "natural" sequence of colors. The HSV (hue saturation value) model uses a color cylinder
 * in which each color is a point inside the cylinder. 
 * The hue is represented by the angle at which the point is, the saturation represents
 * the length (how close to the center the point is) and the value represent the height at
 * which the point is. 
 *
 * By cycling the hue value from 0 to 360 degrees, and keeping the saturation and value at 1
 * we can represent all the brightest colors of the wheel, in a nice natural sequence.
 *
 * The algorithm to convert a HSV value to a RGB value is taken from Chris Hulbert's blog (splinter)
 *
 * Created 1 January 2011
 * By Eduardo A. Flores Verduzco
 * http://eduardofv.com
 *
 * References:
 * http://en.wikipedia.org/wiki/HSL_and_HSV
 * http://en.wikipedia.org/wiki/Color_wheel
 * http://splinter.com.au/blog/?p=29
 *
 *
 */

void setup() {
  //Set the pins to analog output
  pinMode(9,OUTPUT);
  pinMode(10,OUTPUT);
  pinMode(11,OUTPUT);
}

void loop() {
  //The Hue value will vary from 0 to 360, which represents degrees in the color wheel
  for(int hue=0;hue<360;hue++)
  {
    setLedColorHSV(hue,1,1); //We are using Saturation and Value constant at 1
    delay(10); //each color will be shown for 10 milliseconds
  }
}

//Convert a given HSV (Hue Saturation Value) to RGB(Red Green Blue) and set the led to the color
//  h is hue value, integer between 0 and 360
//  s is saturation value, double between 0 and 1
//  v is value, double between 0 and 1
//http://splinter.com.au/blog/?p=29
void setLedColorHSV(int h, double s, double v) {
  //this is the algorithm to convert from RGB to HSV
  double r=0; 
  double g=0; 
  double b=0;

  double hf=h/60.0;

  int i=(int)floor(h/60.0);
  double f = h/60.0 - i;
  double pv = v * (1 - s);
  double qv = v * (1 - s*f);
  double tv = v * (1 - s * (1 - f));

  switch (i)
  {
  case 0: //rojo dominante
    r = v;
    g = tv;
    b = pv;
    break;
  case 1: //verde
    r = qv;
    g = v;
    b = pv;
    break;
  case 2: 
    r = pv;
    g = v;
    b = tv;
    break;
  case 3: //azul
    r = pv;
    g = qv;
    b = v;
    break;
  case 4:
    r = tv;
    g = pv;
    b = v;
    break;
  case 5: //rojo
    r = v;
    g = pv;
    b = qv;
    break;
  }

  //set each component to a integer value between 0 and 255
  int red=constrain((int)255*r,0,255);
  int green=constrain((int)255*g,0,255);
  int blue=constrain((int)255*b,0,255);

  setLedColor(red,green,blue);
}

//Sets the current color for the RGB LED
void setLedColor(int red, int green, int blue) {
  //Note that we are reducing 1/4 the intensity for the green and blue components because 
  //  the red one is too dim on my LED. You may want to adjust that.
  analogWrite(9,red); //Red pin attached to 9
  analogWrite(10,green/3); //Red pin attached to 9
  analogWrite(11,blue/3); //Red pin attached to 9
}

PresentaciÃƒÂ³n en #Demoshow de @Netmedia

Aqui les dejo mi presentación de ayer en el foro Demoshow de Netmedia.

1922 Kodachrome Ã¢Â€Â“ Color Moving Pictures

El color agrega toda una nueva dimensión

http://www.youtube.com/watch?v=J_RTnd3Smy8

TSQL Date String from DateTime

SELECT DATEADD(dd, 0, DATEDIFF(dd, 0, GETDATE()))

via http://blogs.x2line.com/al/archive/2006/02/17/1458.aspx

Post del Equipo de Windows Azure sobre nosotros

Sobre el premio de Netmedia / Information Week

http://blogs.msdn.com/b/windowsazure/archive/2010/07/16/occmundial-named-second-most-innovative-company-with-windows-azure-solution.aspx

Thnx guys @MS!

OCCMundial, Segundo lugar entre las empresas mÃƒÂ¡s innovadoras de MÃƒÂ©xico

El Jueves 24 de Junio asistimos a la entrega de premios "Las 50 Empresas más Innovadoras" de InformationWeek México, donde afortunadamente nosotros, OCCMundial, ganamos el segundo lugar con el proyecto OCCMatch. Muchas gracias a InformationWeek, NetMedia y Mónica Mistretta por el reconocimiento. Gracias a la Dirección de OCCMundial por la oportunidad de innovar, proponer y desarrollar ideas – no en cualquier lugar están dispuestos a correr los riesgos inherentes a la innovación.

En Web, innovar no es opcional: la innovación debe ser una característica más del producto que vendes.

Es por ello que iniciamos OCCMatch hace casi 3 años, recién llegamos a OCCMundial. En principio, OCCMatch fue un algoritmo de estimación de similitud entre ofertas de trabajo y currículos con el fin de correlacionar unos con otros y ofrecer recomendaciones. Empezamos trabajando con el Dr. Aldo Romero de CINVESTAV en Querétaro quien generó la base del algoritmo. Hoy OCCMatch está en vías de convertirse en una plataforma sobre la cual corren diferentes algoritmos (de aprendizaje estadístico, clasificadores bayesianos, filtrado colaborativo, etc.) que ofrecerá una nueva experiencia en la búsqueda de empleo para personas, y de búsqueda de candidatos para las empresas.

OCCMatch corre sobre una implementación, desarrollada internamente, inspirada en MapReduce que corre en la nube de Microsoft Windows Azure, usando más 200 instancias (máquinas virtuales) durante varias horas, diariamente, haciendo match entre las casi 90,000 ofertas activas que tenemos en OCCMundial, y entre 2 y 4 millones de currículos al mes, realizando unas 3.6e11 comparaciones, generando cientos de miles de emails con recomendaciones que son enviados diariamente a los candidatos. Todos los candidatos inscritos al servicio MyOCC son procesados diariamente.

La innovación en el área de tecnología de OCCMundial está comenzando a florecer. OCCMatch seguirá evolucionando. Además, pronto tendremos nuevos servicios para nuestros usuarios y clientes. Los candaditos disfrutarán una mejor experiencia, en un aspecto tan sensible para toda persona como lo es la búsqueda de empleo. Las empresas verán mejorada su eficiencia en la búsqueda de candidatos y sus procesos de Recursos Humanos. Y para nosotros, lo mejor, seguiremos con la emoción y el gusto de seguir innovando y trabajando en lo que nos apasiona.

El efecto de la SelecciÃƒÂ³n Nacional en el TrÃƒÂ¡fico en #OCCMundial: -15%

Una estimación (muy aproximada) del efecto de los juegos de la Selección Nacional en el número de páginas consultadas y visitantes a OCCMundial es de una caída del 15% del tráfico esperado para esa fecha.

Imagínense lo que el Mundial hace por la productividad del país.

4,750 nuevas vacantes al mes en OCCMundial.com

Este año hemos observado un incremento muy importante en el número de Vacantes Activas en OCCMundial, rebasando todos nuestros records históricos. Una Vacante dada de alta o modificada permanece Activa hasta que el reclutador la marca como cerrada o pasa su periodo de expiración (de 7 a 60 días dependiendo de varias circunstancias). El dato de Vacantes Activas se puede ver en las principales páginas de OCC (ver "Contamos con 84,000 ofertas de trabajo" en la barra de búsqueda) y se actualiza cada hora.

En esta gráfica se observa el Promedio Diario de Vacantes Activas y el incremento de vacantes en el último mes. El incremento está dado por la regresión lineal ajustada a los datos (linea azul). En este último mes la tasa de crecimiento fué aproximadamente de 4,750 ofertas, dado por la pendiente de la regresión.

Es importante comentar que el número de Vacantes Activas es un fenómemo muy estacional. En la gráfica se observa el nivel de variación día por por día y semana (incluso por hora las variaciones son notables). Durante el año el mayor incremento se da en el mes de Enero; hay variaciones importantes en días festivos y vacaciones - en la gráfica se observa el efecto del 10 de Mayo - y entre Noviembre y Diciembre cae mucho la oferta de trabajo. Pero eso es tema de otro post.

Alimentos incluÃƒÂdos

En 1889 Max Talmud era un estudiante polaco de Medicina, pobre, viviendo en Alemania. La comunidad judía de Ulm hizo los arreglos necesarios para que las familias locales lo apoyaran de diversas formas. Una de ellas, la familia de Pauline y Hermann Einstein le daría de comer todos los jueves por los siguientes seis años. Durante esas ocasiones, Max presentó a Albert, de entonces 10 años, algunos textos como Elementos de Euclides y La Crítica de la Razón Pura de Kant que fueron influencías definitivas en su formación intelectual.

Wikipedia

6,290,000 Personas en busca de empleo

INEGI acaba de emitir hoy un comunicado de prensa con indicadores provenientes de Encuesta Nacional de Ocupación y Empleo (ENOE) de marzo de 2010.

A partir de la información que da el documento, creo que podemos resaltar varias cosas:

La tasa de ocupación está en 95.19% (desocupación del 4.81%).
INEGI define a la población subocupada como quienes dicen tener necesidad y disponibilidad para trabajar más horas, y ubica esta tasa en 8.57%.
Por lo tanto la población “en busca de empleo” sería la desocupada más la subocupada representa un 13.38% de la PEA (Población Económicamente Activa).
En el comunicado no menciona la cifra de la PEA, pero en el sitio al cuarto trimestre de 2009 la PEA era de 47,041,909 personas según su página de indicadores.
Por lo tanto, la población desocupada sería de aproximadamente 2,260,000 personas y,
la población “en busca de empleo” sería de aproximadamente 6,290,000 personas.

TopManagement sobre OCCMatch

Artículo sobre OCCMatch en TopManagement

Brief on updating Azure WorkerRoles from the July to November SDK version

I'm updating our Windows Azure projects to the new version (November 2009 PDC) of the SDK. These are the highlights on the process:

New namespaces: from Microsoft.ServiceRuntime to Microsoft.WindowsAzure.ServiceRuntime plus the new .StorageClient and .Diagnostics (see below, on logging). Also note that you have to update the References.
The RoleManager class changed to RoleEnviroment so:
- RoleManager.GetLocalResource is now RoleEnvironment.GetLocalResource
- RoleManager.GetConfigurationSetting is now RoleEnvironment.GetConfigurationSettingValue
- RoleManager.WriteToLog disappeared, check the new Diagnostics API
RoleEntryPoint.GetHealthStatus() is gone
RoleEntryPoint changed its execution model from Start->Stop (with the main loop on the Start) to a OnStart()->Run->OnStop() event-like model. Previously I used the class constructor to make the initialization which now goes on the OnStart()

The biggest change for me is the new .Diagnostics namespace. Check this video from the PDC09 to check the new diagnostics and monitoring capabilities.

As for the new .StorageClient, I will not migrate all of our code for now since both C# and Ruby versions runs pretty well. Maybe later.

Cloud Computing

OCCMundial & OCCMatch, our Microsoft Case Study

Yesterday our case study with Microsoft went live here.

It basically explains how we developed our MapReduce implementation on top of Windows Azure to scale the processing of our OCCMatch algorithm. It runs in 200 parallel Azure instances to calculate the best matches for jobs and resumés on OCC, which we send as suggestions via e-mail and are available on My OCC.

Twitter Traffic Hijacking

I've found that some people are using a technique I haven't seen before (which I've called "twitter traffic hijacking") to divert traffic from a known source to their unknown low-traffic site: post a message with the Twitter @user of the known source and invite to follow you with an attractive message.

In this case, user @unlimitedideas is posting a message: "@OCCMundial Tenemos Vacantes, siguenos !!!" (We have job offers, follow us!!!), referencing us OCCMundial (@OCCMundial on Twitter), which is top-of-mind for jobseekers in Mexico, and hoping to appear next to us on Twitter searches. Then they publish messages with their own job offers.

It's certainly ingenious and shows off the way SM is changing the game.

Ã‚Â¿CuÃƒÂ¡ndo se convertirÃƒÂ¡ esto en un problema en Facebook?

Eventualmente mensajes así van a ser un problema:

Charlie falleció hace unas semanas. Fue un buen amigo mientras trabajamos juntos hace 10 años.

SecciÃƒÂ³n Personalizada de Google News para QuerÃƒÂ©taro

Usar: queretaro -gallos -futbol

(Sí hay un pequeño riesgo de perder noticias importantes de gripe aviar, pero vale la pena por evitar información inútil)

Sentido ComÃƒÂºn empezarÃƒÂ¡ a cobrar

El periódico en línea Sentido Común comenzará a cobrar el próximo mes por acceder a sus contenidos. Es una apuesta comprensible en términos de costos pero riesgosa ya que la industria editorial en línea se mueve en sentido contrario, el control sobre el acceso no es tan fácil como parece y vuelve más difícil la indexación del contenido (no se cuánto de su tráfico viene de Google, pero si sigue las tendencias generales en Internet debe ser alto). Yo por lo pronto no lo voy a pagar y seguiré usando opciones gratuitas como Google News.

Demasiados Recuerdos ...

... para un fin de semana:

- la muerte de un antigüo compañero y amigo,

- la casa de 15 años de historia, 4 años de vida, y dos hijos; su primera ropa, los arboles de limón y mandarina,

- las personas con las que viviste 14 años y vuelves a ver tras veinte: amigos entrañables algunos, compañeros agradables muchos, incomprensibles otros,

- saludar nuevamente a una querida maestra.

Yoga y meditación para recuperar el aquí y el ahora.

Square or Circle

A monk asked, "What about when there is neither a square nor a circle?"

Chao-chu said, "Neither square nor circle."

The monk said, "When things are like that, what about it?"

The master said, "Either square or circle."

James Green, The Recorded Sayings of Master Joshu

referenced by Audrey Yoshiko Seo, Enso, Zen Circles of Enlightment

Simple Explanation on Singular Value Decomposition

http://www.ams.org/featurecolumn/archive/svd.html|Geometric approach and samples

Funes and Shereshevskii

I'm currently reading John Medina's Brain Rules (@amazon). Rule #6 Long-Term Memory recalls the case of Solomon Shereshevskii, which reminded my a lot Borges' Funes el Memorioso, although JL regards it as a metaphor of insomnia. (I'm writing this just not to forget)

7 Japanese aesthetic principles to change your thinking

http://www.presentationzen.com/presentationzen/2009/09/exposing-ourselves-to-traditional-japanese-aesthetic-ideas-notions-that-may-seem-quite-foreign-to-most-of-us-is-a-goo.html

Machine Learning Classifier Gallery

http://home.comcast.net/~tom.fawcett/public_html/ML-gallery/pages/index.html|Via HackerNews (discussion)

How a programmer reads your resume (comic)

http://stevehanov.ca/blog/index.php?id=56

Simpson's Paradox

http://en.wikipedia.org/wiki/Simpson's_paradox|(Via Hacker News)

Signs that you're a bad programmer

http://sites.google.com/site/yacoset/Home/signs-that-you-re-a-bad-programmer|Write your checklist

Weird Snippet to Divide an Array [ruby]

Dividing an array of n elements into pieces subarrays is a little bit trickier than it seems. Each subarray will contain chunk_size = n / pieces elements and n % pieces will remain unassigned, and need to be distributed to the subarrays. You can do this in many ways, but in this case we will assign the first remaining element to the first subarray, and so on. So the first n%pieces subarrays will contain n/pieces+1 elements.

def chunk_array(a,pieces)
  cs = a.length / pieces #chunk size
  r = -1*(a.length%pieces) #remaining elements to be asigned to a chunk
  (0..pieces-1).map{|i| r+=1; a[i*cs..(i+1)*cs-1] + (r<1 ? a[r-1..r-1] : [])}
end

R Language Tutorial

http://www.johndcook.com/R_language_for_programmers.html

C# & Linq Functional Style - Simple XML Parsing

With things like this, C# is regaining my attention. Parse a simple XML to a string array containing only the child elements defined in an other string array:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Xml.Linq;

namespace PruebaLinqFuncional
{
    class Program
    {
        public static List<string[]> ParseXML(string xml, string objectName, string[] dataToParse)
        {
            //Programacion funcional en C# :O

            /*
            return XDocument.Parse(xml).Descendants(objectName).Select((element) =>
            {
                var lst = new List<string>();
                foreach (string field in dataToParse)
                    lst.Add(element.Element(field).Value);
                return lst.ToArray<string>();
            }).ToList<string[]>();
             */

            return XDocument.Parse(xml).Descendants(objectName).Select( (element) =>
            {
                return dataToParse.Select( (field) => 
                { 
                    return element.Element(field).Value; 
                }).ToArray<string>();
            }).ToList<string[]>();
        }

        static void Main(string[] args)
        {
            var xml = @"<?xml version=""1.0"" encoding=""utf-8""?>
<EnumerationResults ContainerName=""http://myaccount.blob.core.windows.net/mycontainer"">
  <Prefix>string-value</Prefix>
  <Marker>string-value</Marker>
  <MaxResults>int-value</MaxResults>
  <Delimiter>string-value</Delimiter>
  <Blobs>
    <Blob>
      <Url>Mi_URL_Uno</Url>
      <Name>Mi_Blob_Uno</Name>
      <LastModified>date/time-value</LastModified>
      <Etag>etag</Etag>
      <Size>size-in-bytes</Size>
      <ContentType>blob-content-type</ContentType>
      <ContentEncoding />
      <ContentLanguage />
    </Blob>
    <Blob>
      <Url>Mi_URL_Dos</Url>
      <Name>Mi_Blob_Dos</Name>
      <LastModified>date/time-value</LastModified>
      <Etag>etag</Etag>
      <Size>size-in-bytes</Size>
      <ContentType>blob-content-type</ContentType>
      <ContentEncoding />
      <ContentLanguage />
    </Blob>
  </Blobs>
  <BlobPrefix>
    <Name>blob-prefix</Name>
  </BlobPrefix>
  <NextMarker />
</EnumerationResults>";


            foreach (var stra in ParseXML(xml, "Blob", new string[] { "Url", "Name" }))
                Console.WriteLine(string.Join(",", stra));

            Console.ReadLine();
        }
    }
}

GUIDs in Ruby

sudo gem install guid

require 'guid'

Guid.new

http://rubyforge.org/forum/forum.php?forum_id=4743 go to the bottom)

On Windows this might be useful:

http://www.agileprogrammer.com/dotnetguy/archive/2005/10/27/8991.aspx

StackOverflow Architecture

http://highscalability.com/stack-overflow-architecture|It's good to see that we are doing similar things...

Introduction to LINQ - Simple XML Parsing

http://www.switchonthecode.com/tutorials/introduction-to-linq-simple-xml-parsing|Para comenzar con Linq to XML

Partidos polÃƒÂticos y Web Servers

Para lo que vale, ahora que estamos a unos días de las elecciones y que al fin se callaron las campañas, aquí va una lista de los Web Servers usados por cada partido (afortunadamente ahora ya solo tienen una página de inicio cumpliendo con la ley electoral):

PAN: Microsoft-IIS/6.0 / ASP.NET

PRI: Microsoft-IIS/6.0 / ASP.NET

PRD: Apache/2.2.9 (Fedora) - PHP/5.2.6

PVEM: Apache/2.2.6 (Fedora)

PT: Apache

Nueva Alianza: Apache/2.0.52 (CentOS)

Convergencia: Apache/2.0.63 (Unix) mod_ssl/2.0.63 OpenSSL/0.9.8b mod_auth_passthrough/2.1 mod_bwlimited/1.4 FrontPage

/5.0.2.2635 PHP/5.2.5

PSD: ngnix/0.5.32

The Google Code University

http://code.google.com/edu/|Online courses and tutorials... checking the whole distributed computing resources.

Priced to Sell

http://www.newyorker.com/arts/critics/books/2009/07/06/090706crbo_books_gladwell?currentPage=1

Cool App for Cloud Storage

http://www.codeplex.com/spaceblock| Windows front-end for managing Amazon S3, Nirvanix, Azure Blob Storage, and now Sun Cloud Object Storage

Creating Charts and Graphs with GNU R

http://freshmeat.net/articles/creating-charts-and-graphs-with-gnu-r

AnÃƒÂ¡lisis de Palabras en Ofertas de Trabajo por GÃƒÂ©nero

En OCCMundial.com como parte de nuestro desarrollo de OCCMatch, Abel y yo estamos haciendo un análisis Bayesiano sobre el contenido de las solicitudes de empleo por sexo de los solicitantes. Inspirado en este excelente post de Toby Segaran, les presento un resultado sobre las palabras que, en ofertas de trabajo populares, dan una mayor predictibilidad de recibir un porcentaje de respuesta diferenciado de hombres o mujeres.

En general, esto es lo que hicimos:

Tomar un número grande de ofertas de trabajo populares (que recibieron una cantidad alta de solicitudes)
Dividirlas en categorías de alto porcentaje de solicitudes masculinas, femeninas o ninguna de las dos
Hacer división de palabras y limpieza del contenido, en especial, eliminación de acentos, ñ, etc.
Entrenar filtros Bayesianos para cada categoría
Determinar las palabras que mejor predicen la categoría

Abajo les muestro la lista de las primeras 50 palabras en la categoría de hombre/mujer ordenadas por valor predictivo.

Es interesante que para estas ofertas la diferenciación por sexo se enfoca en puestos administrativos o auxiliares, mientras que en hombres se enfoca en puestos operativos. Llama la atención que en el caso de mujeres el valor predictivo que puede tener una frase como "excelente presentacíon". Otras conclusiones sobre adjetivos o atributos de los candidatos solicitados se los dejo a ustedes ;)

Por supuesto el tema de diferenciación por género en ofertas de trabajo es delicado. OCC ni siquiera tiene un campo para determinar el género solicitado en las ofertas de empleo; pero no se puede controlar el contenido de la vacante que crea cada reclutador. Y es un hecho que en México es una práctica común.

Más Mujeres	Más Hombres
femenino	masculino
excelente	produccion
presentacion	mantenimiento
asistente	sistemas
sexo	industrial
direccion	personal
interesadas	ventas
agenda	ingenieria
conmutador	almacen
office	ing
administrativas	inventarios
tecnica	liderazgo
ley	conocimientos
recepcion	procesos
recepcionista	equipos
proactiva	distribucion
secretaria	gerente
fotografia	ingeniero
trunca	resultados
organizada	calidad
llamadas	conocimiento
ejecutiva	supervision
buena	logistica
comercial	operaciones
preparatoria	operacion
trato	productos
excel	casado
soltera	objetivos
archivo	administracion
proveedores	planeacion
reportes	mecanico
word	seguridad
funciones	soporte
hrs	tecnico
control	obra
buen	proyectos
oficina	manufactura
bachillerato	supervisor
administrativo	rutas
civil	preventivo
caja	producto
apoyo	redes
dinamica	cargo
administrativa	consumo
chica	proactivo
terminada	empresas
fax	mecanica
ortografia	asegurar
general	almacenes
licenciatura	viajar
unicamente	maestria

The Three Sexy Skills of Data Geeks

http://dataspora.com/blog/sexy-data-geeks/|Look for the djvu version of the book

Machine Learning, Artificial Intelligence and Math resources

My Reference List:

A Beautiful WWW - Information Retrieval. Information Visualization. Data Mining. Artificial Intelligence. Web Programming. (by Todd Holloway). Links to a ensemble machine learning tutorial.

[Español]

Dr. Jesús González - Instituto Nacional de Astrofísica, Óptica y Electrónica (INAOE). Ver hasta abajo los cursos y el material disponible
Dr. Enrique Súcar - INAOE. Cursos de Matemáticas, Probabilidad, etc
Dr. Alexander Gelbuk - NLP, libros y papers

The Million Dollar Programming Prize (Netflix Prize)

http://www.spectrum.ieee.org/may09/8788|Via Hacker News

Officially a Eric Lewis fan

I've only heard Eric Lewis here and on the TED Talk but I'm officially fan. This is some of the best new music I've heard on a long time.

How to Choose Colors Everyone Likes

http://www.creativepro.com/article/how-choose-colors-everyone-likes|Design Resources

Los Alamos Center for Bayesian Methods

http://drambuie.lanl.gov/~bayes/index.html|Tutorial, links and publications

Eric Lewis plays chaos and harmony

Amazingly creative and expressive

Don't, The Secret of Self-Control

http://www.newyorker.com/reporting/2009/05/18/090518fa_fact_lehrer?currentPage=all#

Crucial Concepts Behind Advanced Regular Expressions

http://www.smashingmagazine.com/2009/05/06/introduction-to-advanced-regular-expressions/

Dr. Enrique SÃƒÂºcar - Recursos en MatemÃƒÂ¡ticas y Machine Learning

http://ccc.inaoep.mx/~esucar/|INAOEP

Mindfulness In Plain English

http://mail.saigon.com/~anson/ebud/mfneng/mind0.htm|Abajo links a muchos mas libros

Vipassana Center Mexico

http://www.morelos.mx.dhamma.org/

My Guide for Using SQL Server 2005 with Ruby on Linux through ODBC

Update: Links that changed were corrected

First, use this Rails guide to configure ODBC, FreeTDS, RubyODBC:

http://wiki.rubyonrails.org/rails/pages/HowtoConnectToMicrosoftSQLServerFromRailsOnLinux

I also installed the ActiveRecord ODBC Adapter and DBD-ODBC from source

Check this for a hack when DBI can' find the driver:

http://www.ruby-forum.com/topic/162972

Tamiflu $2,000 (US$142) en MercadoLibre... Increible

http://articulo.mercadolibre.com.mx/MLM-28642272-tamiflu-caja-10-capsulas-_JM

Es increible lo que hace la gente. Dejo mi versión congelada para referencia. Ya fue denunciado en Mercadolibre.

Visualizing the Bayes Theorem

http://blog.oscarbonilla.com/2009/05/visualizing-bayes-theorem/

An Introduction to Mindfulness Meditation

http://hivelogic.com/articles/view/an-introduction-to-mindfulness-meditation/

Amazing Design

http://www.webdesignerdepot.com/2009/04/100-amazing-futuristic-design-concepts-w-wish-were-real/

Data & Graphs | Datos y Graficas - Influenza

All data is based on official reports as presented by the media. I'll try to get the references for older data.

Susp: Suspected Nationwide.
Ratio refers to the ratio of suspected deaths of swine flu to total suspected cases on a same report.
Date/Time is aproximately to when authorities made information public.

Todos los datos están basados en cifras oficiales. Voy a tratar de recuperar las referencias de los días anteriores.

Susp: sospechos a nivel nacional; Total:casos, Deaths:fallecimientos
Ratio se refiere a la relación entre muertes sospechosas de H1N1 a casos sospechos dados en un mismo reporte.
La fecha/hora es aproximada al momento en que fueron dados a conocer los datos oficiales

Date	Susp.Total	Susp.Deaths	Ratio	Reference
04/25/09 08:00 PM	1004	68	6.77%
04/26/09 03:00 PM	1300	81	6.23%
04/26/09 08:00 PM	1614	103	6.38%
04/27/09 12:00 PM	1995	149	7.47%	http://www.eluniversal.com.mx/notas/593967.html

Susp.Total | Total de casos sospechos

Susp.Deaths | Fallecimientos sospechos

Ration on reported cases | Relación en casos reportados

Links on Swine Flu (Influenza)

BBC:

News on the Mexico's Outbreak

Q&A

New Scientist:

http://www.newscientist.com/article/dn17025-deadly-new-flu-virus-in-us-and-mexico-may-go-pandemic.html

WHO Containment Plan / Plan de contención de la OMS:

http://www.who.int/csr/resources/publications/SummaryRCProtocol.pdf

Televisa

http://www2.esmas.com/noticierostelevisa/epidemia-influenza-mexico/

Minimalsites

http://minimalsites.com|Beautiful web design

The Monty Hall Problem Simulation

I'll give a small talk about the Bayes' Theorem and some of it uses to my coworkers (presentation here). For that, I've written a simulator of the Monty Hall Problem , which I hope will make fun the talk. Access the simulator here, or via the link on the Sandbox subsection at the right column.

I Told You So - Interview with Nassim Taleb

Check . For me, one of the key teachings of The Black Swan is: the median and other measures are only relevant if the distribution is Gaussian, and it rarely is.

FotografÃƒÂa Aerea de MÃƒÂ©xico - Aerial Photography of Mexico

Afortunadamente sigue mejorando, ahora con fotos del país

startuptools: Hints for Startups. RT@jennielees

startuptools is a nice compilation of tools to&for startups.

Freerisk: Financial Risk Crowdsourced

Interesting new project by Toby Segaran . This is a quite specialized area and knowledge is pretty closed, let's see how crowdsourcing helps.

Awesome music synthetizer

http://lab.andre-michelle.com/tonematrix

Trying Heroku

I'm trying Heroku with the same app that runs this blog. And it's great! In no more that 2 hours I get the app up and running with few workarounds needed.

I had some problems with the tar.gz used to import. It might because a) SVN files and dirs, which I latetly removed, or b) the tarball created on Windows. But I swittched to the Git import method on Ubuntu and everything uploaded ok.
I had Rails 2.1 freezed on vendor/, which after some googling I found it has to be deleted, and comment the line that searches for a specific version of rails.
I also had to remove the freezed version of Ferret and use the one included in Heroku.
The only problem which turned to be impossible to solve was the import process of the data. I used the YAML export as described in the documentation but it failed and sent a message to technical support. My db is MySQL so the problem may be there. Let's see what the guys at Heroku say.

Google AppEngine with Telcel

Telcel is the dominant cell phone carrier in México. Google AppEngine uses a SMS to confirm the registration process but Telcel it's not supported (I can figure out why, since Telcel is not known for it's openess). So that leaved us trying to get some friend with a Telefónica phone to use it to register on AppEngine... until today. I filled the form to report a failure on the SMS service and got a reply from Google saying that they have activated my account. Cool and fast!

HaikÃƒÂº

Agua antigüa

una rana salta

luego, suena

-Basho - mi interpretación

Ten Papers on Information Retrieval and Search

http://www.scienceforseo.com/information-retrieval/10-papers-you-need-to-read/|To check

JRuby on Rails on Google App Engine

http://olabini.com/blog/2009/04/jruby-on-rails-on-google-app-engine/|Cool, maybe will move there

MiIslita

http://www.miislita.com/|Information Retieval, Data Mininning and Search Engines

Understanding TFIDF

http://irthoughts.wordpress.com/2008/07/07/understanding-tfidf/

Startup Marketing Tactical Tips

http://onstartups.com/home/tabid/3339/bid/9008/Startup-Marketing-Tactical-Tips-From-The-Trenches.aspxz

Cool C64 Laptop

Complete story here

Easy AI with Python

http://us.pycon.org/2009/conference/schedule/event/71/

Parsing CSV with Double Quotes using Regular Expressions, C#

I'm pretty sure that code like this adapts better to many conditions, but for a simple, fast way of parsing a CSV with values enclosed by double quotes, use Regex:

using System;
using System.Text.RegularExpressions;

public class Program 
{
	public static void Main()
	{
 	  Show("(\"[^\"]*\")","\"hola\",\"todo,el\",\"mundo\"");
	}

	public static void Show(string r, string s)
	{
	  var ms = Regex.Matches(s,r);
	  foreach(Match m in ms)
	  {
		Console.WriteLine("Groups[0]:{0}",m.Groups[0]);
		Console.WriteLine("Groups[1]:{0}",m.Groups[1]);
	  }	
	}	
}

ASP.NET Test Forms from Remote

<webServices>
  <protocols>
    <add name="HttpPost" />
    <add name="HttpGet" />
  </protocols>
</webServices>

Thanks Mark Brown

Chrome usage, a comparision US-MX

Update 2: Data from Feb.-March'09

Update: I've added a more recent (January '09) statistic at the end.

To complement my last post on the usage of Chrome, I'm posting data from Net Applications referenced by Computerworld, and our data:

Net Applications (presumably worldwide)

Week Starting	Aug.24	Aug.31	Sept.7	Sept.15
IE	72.39%	71.03%	71.24%	71.48%
Firefox	19.54%	19.78%	19.35%	19.42%
Safari	6.27%	6.67%	6.95%	6.73%
Chrome	----	0.67%	0.85%	0.77%

OCCMundial.com (mostly México)

Week Starting	Aug.24	Aug.31	Sept.7	Sept.15
IE	90.88%	90.69%	90.60%	90.39%
Firefox	7.46%	7.32%	7.22%	7.38%
Safari	1.27%	1.26%	1.25%	1.32%
Chrome	----	0.35%	0.54%	0.50%

Update:

From August to January IE is clearly loosing share to Firefox and to a lesser extent Safari and Chrome.

Data from Google Analytics for Dec. 22, 2008 to Jan. 21 2009

Dec.22 to Jan 21
IE	87.48%
Firefox	9.62%
Safari	1.66%
Chrome	0.72%
Opera	0.44%

Data from Google Analytics for Feb. 23, 2009 to March 25 2009

Feb. 23 to March 25, 2009
IE	86.82%
Firefox	10.10%
Safari	1.74%
Chrome	0.78%
Opera	0.46%

The Zen of Creativity

http://www.amazon.com/Zen-Creativity-Cultivating-Your-Artistic/dp/0345466330|Zen and Art, and Zen through art.|

John Daido Loori

I choosed this book for it's compelling title. Although I initially thought that it would be more oriented to show you how to improve your creative skills, it actually guides you to Zen Buddhism through Art. And it's very good at it. Written on a autobiographical way, John explains the basic principles of Zen and how your own artistic and self-expression search can lead you to the discover of the truths of life. Goes between the practice, with even some exercises, to the basics of zazen and the Zen principles, a description of the Zen Arts and the Zen aesthetic.

The Black Swan

http://www.amazon.com/Black-Swan-Impact-Highly-Improbable/dp/1400063515|Randomness and Uncertainity|

Nassim Nicholas Taleb

Excellent. A thoughtful view of the uncertainities of the worlds we live in. Leaves many points to think about and hopefully I'll write about some of them soon. A definitively future reread.

ActiveRecord SqlServer Adapter [Rails,Ruby]

http://github.com/akitaonrails/activerecord-sqlserver-adapter/tree/master

An Intuitive Explanation of Bayes' Theorem

http://yudkowsky.net/rational/bayes|Explanation on Bayes' Theorem

Update: I've just revisited it and is awesome. A great help to understand the problem and maybe even find a way of thinking Bayesian intuitively.

Color Scheme Designer

http://colorschemedesigner.com|Very nice...

Yet Another Machine Learning Blog

http://yamlb.wordpress.com/|To check...

Four from Kandinsky, and Video

Para la Experiencia de Vida en el salón de Andrea, con cariño...

Books and Music That Make You Dumb

http://blogs.wsj.com/digits/2009/02/27/books-and-music-that-make-you-dumb/?mod=yhoofront|Linking to the polemic

Watch the discussion at Hacker News. Check the mirrors for books and music.

Warning Signs in Experimental Design and Interpretation

http://norvig.com/experiment-design.html|by Peter Norvig - A Classic

Related to this post on the Bayes' theorem

No Productive Mediocrity - Leonardo

http://chronicle.com/temp/reprint.php?id=zs61txc4kwr4kd1q1rjbfxt41952gdmf|How to Procrastinate Like Leonardo da Vinci

Odd Crash of Opera/Firefox 3.0 on Ubuntu 8.10

For a time my Firefox was crashing and I couldn't find why or pinpoint which pages made it crash since usually once Firefox was reloaded the pages could be opened. I googled for a while and many reports indicated a problem with the flash plugin and offered several workaround: from reinstalling the adobe plugin to block any flash content. Unfortunately, any of those worked. I got even more puzzled when I started using Opera with similar results. I found a bug report saying that deleting the wins option on /etc/nsswitch.conf may help. Now Firefox seems pretty stable. I'll try to figure out what's happening here and post any update here.

/etc/nsswitch.conf
------------------
#editado por el supuesto bug de firefox https://bugs.launchpad.net/ubuntu/+source/firefox-3.0/+bug/291843
#hosts:          files mdns4_minimal [NOTFOUND=return] wins dns mdns4
hosts:          files mdns4_minimal [NOTFOUND=return] dns mdns4

R language

I've just know R and I'm amazed. Looks great and simple to use in our statistical analysis tasks of the new algorithms we're working on. I've briefly used Octave-Matlab, Python with Numpy and plain Ruby with gnuplot, but R might be what I was looking for.

Installing Monodevelop Beta 2 (1.9.2) from sources in Ubuntu 8.10

Log on my experience installing Monodevelop Beta 2 from sources in Ubuntu 8.10 with Mono 2.2 already installed (the bolded steps are the ones that actually worked).

Install gtk-sharp 2.8.5 with
```
configure --prefix=/usr
```
- FAILED on make from some missing pango, glib depencies
Install mono-addins --prefix=/usr --enable-gui - OK
gtk-sharp 2.8.5 -- FAILS just as the previous
gnome-sharp-2.20 -- Shows some missing gapi-2.0 dependency. Decided try to fix this
Install gtk-sharp-2.12.7 (ignoring some missing things) - OK
Uninstall monodoc de gdeb - OK (after that I found that was not necessary)
Install monodoc 2.0 --prefix=/usr - FAILS to some missing methods. Googling it seems that monodoc in now integrated with mono 2.2 but couldn't find it. Decided to install monodoc 2.0 from apt-get again
Install monodevelop-1.9.2 with ./configure --prefix=`pkg-config --variable=prefix mono` - OK

Accessing a Microsoft SQL Server 2005 Server from Ubuntu 8.10 via ODBC / FreeTDS

Update: Check the encoding conversion note for FreeTDS

Following this post I'm detailing the process I've done (took me less than 10 minutes):

Set up variables on ~/.bashrc - no se si quedaron bien, las hice como root

 export ODBCINI=/etc/odbc.ini
 export ODBCSYSINI=/etc
 export FREETDSCONF=/etc/freetds/freetds.conf

All of the following can be installed via apt-get:

install unixodbc
install tdsodbc
install unixodbc-dev
install freetds-dev

configure /etc/freetds/freetds.conf (check the client charset section since FreeTDS uses ISO-8859-1 by default and it's very likely you want to change it (check here ):

[MY_SERVER_NAME]
    host = 192.168.0.YY
    port = 1433
    tds version = 8.0 #Checar si se usan otras versiones: www.freetds.org
    client charset = UTF-8

install sqsh
test with

sqsh -S DB -U user -P pass

configure /etc/odbc.ini

[MY-SERVER-NAME]
Driver        = FreeTDS
Description     = Conexion a Sql  con FreeTDS / ODBC
Trace         = No
Servername    = MY-SERVER-NAME
Database    = MY-DATABASE-NAME

configurar /etc/odbcinst.ini

[MY-SERVER-NAME]
Driver        = FreeTDS
Description     = Conexion a Sql  con FreeTDS / ODBC
Trace         = No
Servername    = MY-SERVER-NAME
Database    = MY-DATABASE-NAME

test

isql DB SA PASS

the-golden-grid

http://code.google.com/p/the-golden-grid/

Tokyo Cabinet: Beyond Key-Value Store

http://www.igvita.com/2009/02/13/tokyo-cabinet-beyond-key-value-store/|Nice description of this Key-Value store project for Ruby

How Not To Sort By Average Rating

http://www.evanmiller.org/how-not-to-sort-by-average-rating.html

How ASP.NET fails - Ready to Quit ASP.NET

This post by Stefan Rusek express exactly how I feel about ASP.NET from the UI/Web perspective. (Check also the Hacker News thread on it)

I would add that, over the years, the whole .NET framework has grown to a over-engineered beast. It's coherent and consistent, but still a beast. So several parts of the framework simply goes too far on the Object Orientation and builds a excess of classes, realtions, patterns and so on, sometimes making very painful the decision on how to correctly implement things. That's why you feel so liberated when trying some other alternatives like Ruby on Rails (which, I have to say, seems to be growing too much recently).

Using ASP.NET also drives you to blindly use those architectures, patterns and practices, that Microsoft loves, without realizing that usually simpler is better, and without giving a second thought on how and why you are doing what you are doing (LINQ, three tier architecture, and so on). I've seen horrible real-world examples of this.

On the other hand, the .NET framework has some very interesting features like being a complete infrastucture, and C# is evolving in very nice ways, like the new dynamic features. Good things can be done with .NET, but you have to question the "correct ways" as presented by Microsoft.

CSS Palette: Watch the Palette within a CSS

CSS Palette

I've written a quick toy to easily watch the colors used on a CSS. You pass the URL of an online CSS and it will parse the colors and build a palette for you. It's useful if you want to reduce the amount of colors you're using by spotting the almost duplicates that usually arise when writting code.

Color "sorting"

To display the colors in a nice order I wrote a "compare" color function to use with the sort method of the Array. It uses a basic color quantization by calculating the euclidean distance between the color and a origin at (0,0,0) black. Ruby code:

  #compares colors using the position 
  def compare_colors(a,b)
    color_position(rgb(b)) <=> color_position(rgb(a))
  end

  #returns the color "position" relative to (0,0,0) as euclidean distance
  def color_position(rgb)
    #Ruby 1.8.7 ya tiene reduce 
    #Math.sqrt(rgb.reduce{|dist,comp| dist+=comp*comp})
    dist = 0
    rgb.each{|comp| dist+=comp*comp}
    Math.sqrt(dist)
  end

One-line parse of the colors of the CSS

Once the CSS is downloaded with the standard net/http methods, it takes only one line to parse it and build an array of distinct colors (Ruby functional style is awesome):

   r.body.downcase.scan(/(#[0-9a-f]{3,6};)/mi).flatten.uniq

Generating the palette using DIVs

I wanted to show the colors as a single strip of squares that would resize with the browser window. This was a bit tricky because positioning DIV side by side had to be done by creating nested DIVs: the first one is displayed inline which makes possible to wrap the with the size of the window. The second div is displayed normaly as block and has a fixed size of 100x100px:

    <div style="display:inline;width:100%;height:100%">
        <div style="border:1px solid #000;color:<%="#{col[1]}"-%>;background:<%="#{col[0]}"-%>;width:100px;height:100px;float:left"><%="#{col[0]}"-%>
        </div>
    </div>

The CSS Palette is permalinked on the rightbar.

Caching on Rails

http://www.railsenvy.com/2007/2/28/rails-caching-tutorial|How current this info is?

Blogging desde Wii

Escribiendo desde el Wii. Bien en general solo hay que desactivar el Tinymce.

ImageMagick and RMagick on Vista

The current version of the Ruby Gem RMagick (2.9.0-x86-mswin32) comes bundled with ImageMagick installer 6.4.8-6-Q8-windows-dll which is pretty new. I have had some problems after installing both as the instructions indicated with my rails app as Mongrel throwed some CORE_RL dll not found, so I uninstalled both and reinstalled ImageMagick running the exe As Administrator, and everything seems ok now...

Ferret on Windows [Ruby]

If you are installing Ruby indexer Ferret on Windows, gem version 0.11.6 seems broken... install version 0.11.5

gem install ferret --version 0.11.5

Tools

Tools I always install:

Vim
WinMerge: Diff & Merge, with the next two [Windows]
TortoiseSVN: Makes easier SVN and is required for the following [Windows]
VisualSVN: To integrate SVN to Visual Studio [Windows]
Launchy: Makes you way more productive [Windows, testing on Linux]
IMy: Integrates Live Messenger with Launchy. I'm thinking on extending it to Pidgin for Linux.
Ext2 IFS for Windows: Access ext2/3 filesystems on Windows for dual boot environments
7Zip
PuTTY and WinSCP
Cygwin
ViEmu: Vim emulator for Visual Studio.
GIMP and Inkscape

Firefox addins:

Firebug
YSlow
Clear Cache
Cooliris

Recreating the Button

http://stopdesign.com/archive/2009/02/04/recreating-the-button.html|Nuevos botones de Google

Ruby 1.9.1 / 1.8.7 "dual" mode on Ubuntu

I think this may be the most convenient way to have the shinny new Ruby 1.9.1p0 to play and a normal Ruby 1.8.7 on Ubuntu:

Install Ruby 1.8.7 and everything else from apt/synaptic
Download and extract Ruby1.9.1
Use --program-suffix=1.9 while configure to "tag" all the executables

tar jxvf ruby-1.9.1-p0.tar.bz2
cd ruby-1.9.1-p0/
autoconf
./configure --prefix=/usr/local --program-suffix=1.9 --enable-pthread --enable-shared
make
sudo make install
ruby --version
ruby1.9 --version
irb
irb1.9

Try Time.now.to_s inside irb to check differences.

Reference: http://programblings.com/2008/11/18/installing-ruby-19preview1-on-os-x-leopard/

Data Visualization with Ruby

http://www.omninerd.com/articles/Automating_Data_Visualization_with_Ruby_and_Graphviz|I'll check this later, looks interesting

Inductio Ex Machina

http://mark.reid.name/iem/|Blog on Machine Learning. Really good

Academic Earth

http://academicearth.org/|Justo lo que hacía falta

Ubuntu 8.10 on XPS m1530

- Touchpad: Had to use the .8042.nomux=1 boot param for Kernel as described everywhere.

- Problems with the webcam: Cheese and Camorama useless but VLC and Ekiga ok. Seems to be some GStreamer problem. Update: new version of cheese 2.42.2 now works

- Dual screens were quite quirky to configure. Use gksudo nvidia-settings, Twin display mode and set the layout for both screens.

- I'm having some problems with sound. Sound is very low although everything is set up and loud. The master volume control for ALSA mixer sounds no-linear. There seem to be some problems with the drivers, but for now I'll leave things like that.

- Touchpad too sensible. Fixed installing GSynaptics and enabling it as in here. (The issue is hard to test since appears as a "crazy mouse" when typing)

References

https://wiki.ubuntu.com/InstallingUbuntuOnADellXPSM1530

http://jesperdj.pbwiki.com/Ubuntu-on-the-Dell-XPS-M1530

Key to understand Google Analytics

http://www.google.com/support/googleanalytics/bin/answer.py?answer=57164&cbid=1m0o6pg5yvuwv&src=cb&lev=answer

Mounting Windows (CIFS) drives in Ubuntu

http://ubuntuforums.org/showthread.php?t=288534

Compiling Mono 2.2 from souces in Ubuntu 8.10

Update: The original link is gone now. I found a clone and Freezing it.

In general, follow this but keep in mind:

My installation path is to /usr (not /usr/local).

To install libgdiplus: use X11 headers installing libx11-dev, and libfontconfig1-dev, libxft-dev

Anatomy of a Program in Memory

http://duartes.org/gustavo/blog/post/anatomy-of-a-program-in-memory|Excellent post on how operating systems handle memory. A good refresher.

Video de Andrea (2) - Reciclaje

Reciclaje en Yahoo! Video

Operating System usage in OCCMundial.com

This is a brief statistic of the Operating System usage to access OCCMundial.com over the last month, from December 22, 2008 to January 21, 2009 as reported by Google Analytics. The Operating Systems, with less or equal than 0.05% of the visitis and that are not mentioned in the table, are (in decrement order): SymbianOS, Playstation 3, Nintendo Wii, SunOS, PalmOS, PSP, FreeBSD, Android, Danger Hiptop (?), and OpenBSD (and others not identified).

It's worth to mention that in this period of time we had over 5 million visits, so we consider this a pretty good sample of the overall usage of Internet in México.

OS	% of Visits
Windows	97.64%
Macintosh	1.65%
Linux	0.35%
iPhone	0.14%
iPod	0.12%

Dilbert's new company

Rails on Ubuntu 8.10: require_frameworks openssl error

http://www.ruby-forum.com/topic/136740

Just install libopenssl-ruby package

Vista "Open Command Prompt Here"

http://blogs.msdn.com/tims/archive/2006/09/18/windows-vista-secret-1-open-command-prompt-here.aspx

It doesn't work exactly as the old powertoy. You have to right-click inside the folder (that's on the right pane of Windows Explorer) and not over the folder in the Directory browser. I like more the old powertoy.

DIY DNA: One Father's Attempt to Hack His Daughter's Genetic Code

http://www.wired.com/medtech/genetics/magazine/17-02/ff_diygenetics?currentPage=all

Even if it's absolutely improbable that Hugh's quest turns successful, he MUST go on. He is in the land of uncertainity, with a huge payoff if he succed: his daighter's health. That reward is so great that even with a extremely low chance, it's worth trying. This is one of the most inspiring stories I've ever read and send best wishes to Beatrice, Hugh and his family... I hope there's anything we can do to help.

Collection of papers on introductory web mining

http://www.informatics.indiana.edu/fil/Class/b656/schedule.html

How to create a profitable Freemium startup

http://andrewchenblog.com/2009/01/19/how-to-create-a-profitable-freemium-startup-spreadsheet-model-included/|Para leer después

GIMP language change

GIMP is great but I hate that sets the language to the current locale ignoring even Windows.

GIMP 2.6 path to locales is c:\program files\gimp-2.0\share\locale, and yes, delete all but en-* locales.

http://www.gimptalk.com/forum/viewtopic.php?t=1994

Ruby Linalg

http://rubyforge.org/projects/linalg/|Fortran based Linear Algebra Library for Ruby

The History of Python

http://python-history.blogspot.com/|By Guido

15 minutes to using your existing Windows install & apps in Ubuntu

http://74.125.45.132/search?q=cache:http://www.venturecake.com/a-simple-guide-to-using-your-existing-windows-install-apps-in-ubuntu/|(Cached on Google)

Hallucinations

http://www.boston.com/bostonglobe/ideas/graphics/011109_hacking_your_brain/|How-to-Hallucinate

Markov Chain Monte Carlo sampling toolkit.

http://pypi.python.org/pypi/pymc/

Compilation of Feynmann's Lectures

http://freescienceonline.blogspot.com/

Reseting the Video Settings the Wii

If you kept the video setting from widescreen/480i and moved the Wii to a CRT, here's how to reset them:

- Power on Wii, give it a few seconds to boot
- Hold DOWN on the directional pad of the Wii remote
- While holding DOWN, press the RESET button on the Wii

The Wii reboots and the screen settings are back to default. All my other settings (wifi, console name, etc.) were intact.

Link (look for ch0mp's comment). Also, you can switch to the A/V and the go back to the composite cables.

Google Native Client on Spanish

Use this workaround to get Google Native Client on a Spanish Windows XP w/Firefox 3

The physics of networks

http://ptonline.aip.org/journals/doc/PHTOAD-ft/vol_61/iss_11/33_1.shtml?type=PTALERT Algoritmos|Physics|Networks

A few moments ago my Chrome refreshed http://maps.google.com/help/maps/elections/index.html#2008_election showing 324 vs 124 votes favoring Obama. I'm also mourning for México, on the airplane crash and death on Secretary Mouriño. Strange days.

Linear Algebra for Free

http://www.randomhacks.net/articles/2007/03/07/hefferon-linear-algebra-review

Beta de OCCMundial.com al aire

El nuevo Beta de OCC ya está al aire... comentarios en el sitio o aquí mismo

http://beta.occmundial.com

Si se puede, luego postearé algo de la nueva arquitectura, que no es por nada pero es bastante novedosa :D

Ruby: Error with Gems 1.2.0 + Win32_Process + $KCODE + DRb

This is really awkward. The problem was found after moving some scripts that worked fine on one server to another server with rubygems recently updated. The script normally spawns several processes that are DRb Servers, but it failed silently. After some testing I found that with every other thing being equal (Ruby 1.8.6 patchlevel 111 and win32-process 0.5.3 on both machines) the script crashed with gem 1.2.0 and not with 0.9.4.

The script iterates calling Process.fork with a code block that creates the new DRb server. The DRb object is loaded via require at the begining of the script. I'm posting here a reduced example for clarity.

puts "iniciando"
require 'rubygems'
require 'drb'
puts 'loading pingpong'
require 'pingpong'
puts 'loaded pingpong'
puts 'loading process '
require 'win32/process'
puts 'loaded process'

def start_server(port)
  uri="druby://0.0.0.0:#{port}"
  trap("INT"){puts("Interrupted"); DRb.thread.exit}
  DRb.start_service(uri,PingPong.new)
  puts("Listening #{uri}")
  DRb.thread.join
end 

puts "here #{ARGV.inspect}"
2.times{|port| puts("Sending #{port}");Process.fork{start_server(5850+port)}}
puts "out"

The PingPong class is defined in the following script. Note the $KCODE declaration since it turned out to be the problem:

$KCODE = 'UTF-8'

class PingPong
  def ping
    "pong"
  end
end

In the current implementation of win32_process, Process.fork works by creating a new Windows process calling Ruby with the same script and passing an argument indicating which child is creating. This is very important as indicated in the documentation since one would expect that each child's execution started at the code block passed to the fork.

The process failed like this with gem 1.2.0 (but worked perfectly on gem 0.9.4:)

C:\>test_fork_simple.rb
iniciando
loading pingpong
loaded pingpong
loading process
loaded process
here []
Sending 0
Sending 1
out

After some hours debugging I found that the problem arised when the $KCODE assigment is done on the child processes; if the assigment is deleted the processes are created correctly:

C:\>test_fork_simple.rb
iniciando
loading pingpong
loaded pingpong
loading process
loaded process
here []
Sending 0
Sending 1
out

iniciando
iniciando
loading pingpong
loading pingpong
loaded pingpong
loading process
loaded pingpong
loading process
loaded process
here ["child#0"]
Sending 0
loaded process
here ["child#1"]
Sending 0
Sending 1
Listening druby://0.0.0.0:5850
Listening druby://0.0.0.0:5851

But in the real programs the line can't be left out since it's critical. Oddly enough, the workaround consisted on switching the 'require' statments for the child class and win32/process:

#Crashes silently:
require 'pingpong'
require 'win32/process'

#Works!
require 'win32/process'
require 'pingpong'

It's also very strange that, when the script fails, the child processes are not even created (as you can see on the first output, the "iniciando" message is never sent). I'll post an update if I found what's happening here.

Video de Andrea

Video de tarea de Andrea

Economia de Tailandia (Kinder 3A) @ Yahoo! Video

Loading and Processing Big Matrices on Python/numpy for NNMF

I was loading a quite big dataset (3446x14807 floats) to a Python / Numpy matrix to perform a Nonnegative Matrix Factorization (NNMF). First I tried generating the Python source code from another process and inserting literally the whole matrix, something like:

w = matrix([
[0.0072992700729927,0.0072992700729927,0.0291970802919708,0.0145985401459854,
0.0072992700729927,0.0072992700729927,0.0072992700729927,0.0072992700729927, ... ,])

But the process crashed with a cryptic MemoryError while loading the matrix. Then I tried using

loadtxt()

Numpy's method but the same thing happened. I found this thread. I'm using both a XP laptop and a Win2003 Server with Python 2.5.1 and 2.5.2 respectevly, both 32bits. This didn't seemed the problem since the matrix could be constructed with ones or zeros and enough memory was available (the server has 8gigs of RAM). It rather looked like the problem was while building the matrix. So I found that building the array first and then adding the values did the trick:


JW = zeros((len(jobs),len(words)), float)
f=open(argv[1]+'_JW.csv')
line = 0
for l in f:
  vals = l.split(',')	
  for i in range(0,len(vals)):
    JW[line,i] = float(vals[i])
  line += 1
f.close()

Crisis = Danger + Opportunity

Update: Check this interesting clarification by Victor H. Mair. This image is now pointing to Victor's.

TED talk on Morality and Political Stands

Insightful. Check also the Morality Quiz

Chrome #4 on OCCMundial.com

Google Chrome ranks #4 with 0.51% of visitis on OCCMundial(my employer) as of today, from Sept 2. It ranks over Opera (0.27%) and below Internet Explorer (90.54%), Firefox (7.29%) and Safari (1.26%). OCCMundial is the leading online Job Board in Mexico, with more than 100 million pageviews per month.

And yes, it's noticeable the margin of IE over every other browser, which I think can be extrapolated to the whole usage of browsers in Mexico since our traffic it's pretty high for this market. (Data from Google Analytics).

On Artistic Taste and Psychology

How psychology accounts for taste development Via Psychology Today

"We wrongly assume, for instance, that people with highly decorated and cluttered rooms are more extroverted. We also assume such people are more open—when really we should be looking for variety in books and music, for books on art and poetry, and for art supplies. We assume that rooms with stale air belong to emotionally unstable people—when really we should be scanning for inspirational posters"

On Peter Donnelly Conference at TED [Ruby]

A small Ruby script to test Peter Donnelly's example on uncertainity. Conference below..

def avg_pat(pat, t, s=0)
  t.times do 
    last = ''
    last += (rand(2)==0 ? 'H' : 'T') while last[-pat.length..-1] != pat
    s += last.length 
  end
  s/t
end	

['HTH','HTT'].each{|p| print "#{p} #{avg_pat(p,10000)} "}

Link: Peter Norvig at Startup School 2008

http://omnisio.com/startupschool08/peter-norvig-at-startup-school-08|Great talk of Peter Norvig

IronPython: Catching exceptions

Don't know why this wasn't easy to spot. To catch an exception and display the StackTrace on IronPython use:

import sys

try:
	print "hola"
	print 3/0 #Select which exception to test
	raise Exception("Error...")
except Exception, e:
	print e.clsException

Ruby: Awsome Collection of Ruby Tricks

http://www.rubyinside.com/21-ruby-tricks-902.html

Rails: Fast & Easy Sitemaps

Few lines to get your sitemap. Also, add a hash array for the other links on your site:

Sitemap controller:

    @sites = [
	    {:loc=>url_for(:controller=>'home',:action=>nil), :lastmod=>File.open('web/app/views/home/index.rhtml').mtime.strftime('%Y-%m-%d')},
	    {:loc=>url_for(:controller=>'blog',:action=>nil), :lastmod=>File.open('web/app/views/blog/index.rhtml').mtime.strftime('%Y-%m-%d')},
	    {:loc=>url_for(:controller=>'casa',:action=>nil), :lastmod=>File.open('web/app/views/casa/index.rhtml').mtime.strftime('%Y-%m-%d')}
    ]

RXML:

  @sites.each do |site|
    xml.url do
      xml.loc( site[:loc] )
      xml.lastmod( site[:lastmod] )      
    end	    
  end

He de ser estÃƒÂºpido

En la página del Gobierno del D.F. está la declaración patrimonial de Marcelo Ebrard: en resumen, el año pasado contaba con un patrimonio de menos $300,000, un solo coche y ningún bien inmueble propio (eso sí, algunas obras de arte). A la fecha, cuentas bancarias por menos de 1 millón y sin otros cambios. Si Marcelo cree que soy estúpido, he de serlo...

CSS: Wrapping blocks of code

Cool... [check this]

La peor ignorancia

La peor ignorancia es no saber que no sabes.

Ruby: Change file access/modification time for a file

Pretty basic, but took me one hour to find. To set the access/modification time for a file in Ruby do not try to use

FileUtils.touch

(as Unix) because it only sets to current DT. Use

File.utime(atime,utime,filename,...)

Finding the Still Point

http://www.amazon.com/Finding-Still-Point-Book-Communications/dp/1590304799|The essence of zazen|

By John Daido Loori

Very simple & beautiful approach to zazen. Great as start point.

Zen Mind Begginer's Mind

http://www.amazon.com/Zen-Mind-Beginners-Shambhala-Library/dp/1590302672|"The purpose of studyng Buddhism is not to study Buddhism but to study ourselves."|

By Shunryu Suzuki

The Tipping Point

http://www.amazon.com/Tipping-Point-Little-Things-Difference/dp/0316346624|Epidemics at Large|

by Malcom Gladwell

The basic premise on this book is that epidemics can be thought as a general pattern appliable as well to health, social phenomena or technology. A very well written book, Gladwell makes a extensive investigation, gathers information that may seem disgragated and elaborates a proposition. Not a scientific method traditional approach, but a good starting point to the understanding and use of this knowledge.

Wabi-Sabi: for Artists, Designers, Poets & Philosophers

http://www.amazon.com/Wabi-Sabi-Artists-Designers-Poets-Philosophers/dp/1880656124|Imperfect Impermanent Incomplete|

By Leonard Koren.

Wabi-Sabi is the quintessential aesthetic of the Japanese art. This gorgeous book describe what is Wabi-Sabi in a Wabi-Sabi format and content. In less than 80 pages, many filled with beautiful photographs, describe the metaphysical, spiritual, state-of-mind, moral and material qualities that create Wabi-Sabi.

The Wabi-Sabi quality can be discovered everywhere, although demands careful attention to find it. For us, quite far from the Japanese mind, Wabi-Sabi explains the beauty we feel and see on many things that are nevertheless overlooked and unappreciated. The tension between the Wabi-Sabi aesthetic and our concious western-modernist aesthetic might be extremely interesting to explore.

Again, again and again... The Little Schemer

Getting back to The Little Schemer proved more challenging. The last couple of chapters are pretty hard and will need a revisit ASAP. The presentation of the Y combinator as a consequence of recursion over the generation of functions is mind-blowing.

This book make me feel that I know nothing about computing and need to learn everything again. And it's great!

The Culture Code

http://www.amazon.com/Culture-Code-Ingenious-Understand-People/dp/0767920562|Clotaire Rapaille|

Dr. Rapaille describes in this book his discoveries on Archetypes applied to marketing. Gives a very general description of his method to uncover them using focus groups in extended sessions. One of the dissapointments about this book was my expectation to understand different cultures as it focus absolutely on the United States and a few topic on the French, English and German cultures. Some peeks to the Japanese and Italian. But that's it. Patethically Euro-American centered. Also, it's quite dissapointing the feeling left behind the attitude of being "more American than the Americans" o más gringo que los gringos, as the most common words are US, WE, OURS... It gets to a point in which is hard to say if an archetype, or "Code", is mostly a reflection of Dr. Rapaille's feelings.

Anyway, the insights given by the book are very interesting, as the sex/violence, food/fuel, money/proof... Most Codes are instructing and give us a peek on the perceived ingenuity, greed and results oriented attitudes we have of the americans.

[Hope I can extend this post as I re-scan the book]

Dunning-Kruger effect and Creativity

I've recently found about the Dunning-Kruger effect on a blog post by Steve Yegge. Despite the fact I do like Joel's book and even have used his technique on a few recruiting process, I've found that for everyday businesses, even Smart&Get-things-done guys can be too hard to get.

Anyway, the point I want to talk about is a side effect of Dunning-Kruger. While in college, in some classes, I would get to a point in which I knew almost nothing about something but was very creative about it, getting lot's of ideas to do things that I didn't knew in that moment were hard or impossible to do. The DK effect. But some of those ideas were worth trying for the learning. And even some of them, although they would be harder to execute than it initially appeared, were good ideas indeed. Because of DK, you don't know the limitations and thus your brain is free to create. When you get to know a lot about something, limitations and restrictions are evident and thus your brain restrain some of those ideas. Anyway, the truth is that some of those ideas could be accomplished and be good.

Once, on a Digital Image Processing class, I had the idea of processing photographs of rooms, aisles, buildings, etc along with some makers. Then make a 3D projection of the markers to "infere" the dimensions of the room, making measurement of spaces simple as a click. At first, the idea sound great, and started working on the math. Then I passed through a disenchantment phase: it was like 1994, digital cameras weren't readily available, the math started to be harder than I knew and computing processing was not so easy to get. Then I quit the idea and moved forward. Now, I know better and maybe it was a good idea. I've seen some 3D projections based on 2D images and are awesome.

That said, I will give a chance to those naïve ideas. Maybe someone could be good after all...

Easiest Permalink/PrettyURLs on Rails

This is the simplest way I've found to get pretty permalinks and/or URLs for this blog (for SEO optimization): use a 37signals-like standard prefix the id of the post to the prettyfied version of the title of the blog. Use the following to Find the post:

def self.find_by_permalink(permalink)
  return(nil) unless permalink
  self.find(permalink.split("-")[0])
end

Use the following to build the permalink:

def permalink
    "#{self.id}-#{self.title.gsub(/[^a-z0-9]+/i,'-')}"
end

The Little Schemer

http://www.amazon.com/Little-Schemer-Daniel-P-Friedman/dp/0262560992|Learn to think in Scheme|

This is one of the best programming books ever. Refresh your mind by learning to think again.. this time, in Lisp.

Truncate HTML output on Ruby (and Rails)

For this little blog app, I needed to truncate the post length on the home. Of course it can't be done by a simple truncate since it can break HTML tags. I found this awesome hack by Mike Burns.

Vendo mi casa

http://www.fusionmxsoftware.com/casa|en México D.F.

TED

http://www.ted.com|Ideas

ERB with custom tag

Ruby ERB is a great library that let's you process a file and embed Ruby code within it using <% ... %> tags. As the file is processed, the code is evaluated and content may be replaced.

This is generaly used with HTML (or RHTML as generally known). Rails does this.

I'm now coding a generator for ASP scripts using ERB. As both ERB and ASP uses the <% <%= tags, I modified the original erb.rb file to use <@ <@= tags so the ASP code can be left as usual. Get the modified file here. If you want to use a diffent tag, I recommend to diff my file with the original erb.rb

How to REALLY disable ASP.NET viewstate

The .NET Framework and ASP.NET have some aspects that sometimes really annoys me. One of these is the infamous __VIEWSTATE used to track the state of all the controls in the page. What is annoying about it is that it’s both a security breach and a performance issue. Also, makes easy things hard.

I wanted to remove it from an app I’m working on, so I started disabling it as suggested by MS. The VIEWSTATE is effectively not used but it stills yields the hidden input field to detect the Postback. This is a nag since this form is performing a GET action and the __VIEWSTATE is passed as part of the QueryString. After some Googling I found a method suggested by Scott Hanselman to move the position of the input field. I just adapted to plainly remove the complete thing as shown below:

       protected override void Render(HtmlTextWriter writer)
        {
            var sw = new StringWriter();
            var htmlWriter = new HtmlTextWriter(sw);
            base.Render(htmlWriter);
            string html = sw.ToString();
            int start = html.IndexOf("<input type=\"hidden\" name=\"__VIEWSTATE\"");
            if (start > 0)
            {
                int end = html.IndexOf("/>", start) + 2;
                html = html.Remove(start, end - start);
            }
            writer.Write(html);
        }

}

All this code just to get a simple, common, standard query string.

Yet another MapReduce implementation in Ruby

^{This post is on construction.}

I'm working on a Recommendations engine. As some processes may be lengthy, I'm also implementing a simple MapReduce framework using a Ruby/DRb schema. I tried Skynet for a while but it looks inmature, and I needed something simple and fast to implement.

The Server - Worker

The DRb server was pretty simple to implement. I just wrapped my Recommendations engine on a single class and created a function for starting it as a DRb server:

def start_server
  uri="druby://0.0.0.0:#{ARGV[0]||'5850'}"
  trap("INT"){puts("Interrupted"); DRb.thread.exit}
  DRb.start_service(uri, RecommendationsServer.new)
  puts("Listening #{uri}")
  DRb.thread.join
end

Just run the server on every processor to which you can distribute some processing load.

The Client - Controller

Next, I crafted a simple program that implemented the MapReduce model by segmenting an array with the data to process, distribute the segments to the workers and yielding the disered operation (which must be defined on the worker server class).

#map
# data  : hash to be mapped to @workers
# worker_object and segment are yielded
def map(data)
  #data = @apps #version lineal (sin procedimiento)
  threads=[]
  result={}
  inicio = 0
  #sec_len = data.keys.length / @workers.length #cargas iguales
  @workers.each do |worker_uri,porcentaje_carga|
    threads << Thread.new( 
          #:prefs=>prefs, #carga igual 
          :segmento=>segmento(data.keys,inicio,porcentaje_carga),
          #:selection=>@apps.keys[worker_n*sec_len..(1+worker_n)*sec_len], #carga igual entre procesos
          :worker_uri=>worker_uri) do |p|
      msg("Spawning worker with #{porcentaje_carga}% load @ #{p[:worker_uri]} ")
      worker_object = DRbObject.new(nil,p[:worker_uri])
      result["result_#{worker_uri}"] = yield(worker_object,p[:segmento])
      #result["result_#{worker_uri}"] = worker_object.calculated_matrix(:prefs=>p[:prefs], :selection=>p[:selection], :type=>:similar, :print=>true)
    end
    inicio += long_segmento(data.keys.length,porcentaje_carga)
  end
  threads.each{|t| t.join}
  return result
end

def reduce(result_set,output_filename=nil,result_set_name=nil)
  r = {}
  result_set.each_value{|worker_result| r.merge!(worker_result) if worker_result}

  if output_filename
    File.open(output_filename,"w") do |f|
      f.write("#{result_set_name||'result'}=#{r.inspect}")
    end
  end

  return r
end

Simplify the process of calling a whole MapReduce cycle:

#args:
#  :prefs=> conjunto de datos
#  :type=> tipo de matriz a calcular
#  :output_filename => archivo de salida
def map_reduce(args)
  msg(args[:process_name]) if args[:process_name]

  #map
  start = Time.now
  msg("Mapping...")
  result_set = map(args[:data]){|worker_object,segmento| 
    #worker_object.calculated_matrix(:prefs=>args[:data],:selection=>segmento,:type=>args[:type],:print=>:true)
    yield(worker_object,segmento)
  }
  msg_ok(start)

  #reduce
  msg("Reducing...")
  start_red = Time.now
  r=reduce(result_set,args[:output_filename],args[:result_set_name])
  msg_ok(start_red)

  msg_ok(start,"#{args[:process_name]} finished") if args[:process_name]
  return r
end

Finally, calling the MapReduce complete process is as simple as:

  ub_recs = map_reduce(
    :data=>@apps,
    :process_name=>'Calculating user based recommendations',
    :output_filename=>'__ub_recs.rb',
    :result_set_name=>'@@ub_recs') do |worker_object,segmento|
      worker_object.calculated_matrix(:prefs=>@apps,:selection=>segmento,:type=>:recommendations,:print=>true)
  end

The yield keyword passes the segmented data and operation parameters to the worker process.

Later, I'll post on how to load-balance between workers, which proved to be both critical and trickier than first apperared.

Firefox 3/2 side-by-side on Windows

Want to use FF2/3 side by side? Check this. Although it describes the procedure for Mac/Linux, everything is almost the same for Windows. Just check this recommendations:

Download the *.win32.zip from the night build.
Unpack on a new directory, say \Program Files\Mozilla Firefox 3
Create the new profile as described
Create a shortcut to \Program Files\Mozilla Firefox 3\firefox.exe with options:
```
 -P newprofile -no-remote
```

That's it!

On Ruby & Python Performance

I'm working on some algorithms taken from the amazing book "Collective Intelligence" by Toby Segaran. Some of these algorithms can be pretty computing intensive, so I'm working also in _yet another implementation_ of MapReduce on Ruby/DRb. So I had to re-implement Toby's Python algorithms on Ruby, which pretty straightforward.

This is a little chart of the processing time of one of the algorithms running on different languages/implementations with the same data subset.

Language	Version	Time
Ruby	1.8.6	16.6 seg
Python	2.5.4	4.15 seg
IronPython	1.1.1	10.45 seg
Ruby	MapReduced	10.65 seg

Pruebas

Nuevo Blog

Moviendome nuevamente, del viejo blog en "Blogspot":http://eduardoflores.blogspot.com Ã‚Â¡Espero que este funcione mucho mejor!