(951) 268-7836 info@authintel.com

SQL Server 2016 RTM

Some major enhancements and a convergence on features from Azure to on-premise. The ‘R’ capabilities and seamless integration with big data are game-changers. https://www.microsoft.com/en-us/cloud-platform/sql-server...

When two-factor authentication really isn’t

My hope is that by now all companies are using multifactor (MFA for short) authentication to protect all critical assets. MFA authentication simply means that a user must validate their identity through at least two independent mechanisms. Two-factor authentication or 2FA for short just means two independent mechanisms are utilized. However many companies that are implementing 2FA are not implementing true 2FA because the mechanisms are not totally independent. Unless there is no way that one of the mechanisms in a 2FA scheme can be leveraged to gain access to the other mechanism, the mechanisms cannot be considered independent. Two common 2FA schemes that are subject to compromise because of lack of independence are the use of a dial-back number that goes to a smart phone and hosting a 2FA on a corporate laptop. This post focuses on the risk with a smart phone that is on the receiving end of a dial-back number. Here is a typically sequence of events for 2FA involving a dial-back number: 1) Logon to a web site or remote desktop or some other resource 2) User is prompted for credentials 3) User enters credentials 4) Upon successful entry of the credentials, the system prompts for an access code that is communicated on the dial-back number. 5) User then receives a phone or in the case of a cell, perhaps a text message or there may be an app configured to acknowledge the call. 6) At this point, the authentication generally occurs in either of the below fashions: a. User enters the code communicated via text message or voice call to the primary authentication...

Multiple divergences for S&P analyzed (Corrected)

Note: My first version of this had an error in the query which has been corrected and the difference between up and down days is not significant. However, the larger the VIX change, the more likely to be a VIX up day and S&P down day the next day. Today was an interesting day in the stock market. The S&P 500 closed positive while the most correlated indicators aside from other equity indexes closed strongly negative.  This includes Treasuries (up) and high yield/corporate bond (down). These are all the opposite results of the norm, at least for the last several years. Along with that volatility was down significantly, which does correlate with upwards moves. Since this seems to make the move in the S&P suspect in terms of at least the immediate term, I ran a query in my equity history database. I used the cube with the grouping function to get a rollup of each grouping and the entire selection.  Below is the query: Although I have history of the S&P back to 1950, some of the other instruments only go back to 1990, so the analysis is limited to then. 195 instances were found that met the condition over a 25 year period, so this only has only happened on average about 13 times per year.  Results for Days where VIX is down while High Yield is down, Treasuries are up, but S&P 500 is up: Out of 195 instances, VIX was up the next day slightly less often (93 versus 102 instances) test.  Along with that, the average change on the days that the VIX went...

Multiple divergences for S&P analyzed

Today was an interesting day in the stock market. The S&P 500 closed positive while the most correlated indicators aside from other equity indexes closed strongly negative.  This includes Treasuries (up), high yield/corporate bond (down), and Volatility (down). These are all the opposite results of the norm, at least for the last several years. Since this seems to make the move in the S&P suspect in terms of at least the immediate term, I ran a query in my equity history database. I used the cube with the grouping function to get a rollup of each grouping and the entire selection.  Below is the query: Although I have history of the S&P back to 1950, some of the other instruments only go back to 1990, so the analysis is limited to then. 152 instances were found that met the condition over a 25 year period, so this only has only happened on average about 6 times per year.  Out of 152 instances, VIX was up the next day almost 1 1/2 times as often (61% versus 39%) based  on 93 out of 152 instances positive for the test.  Along with that, the average change on the days that the VIX went up was negatively much stronger than on the VIX down days. The up days occur most often during tumultuous markets such as 2000 and 2007. Below are the detailed results. Are 152 instances enough data points to justify making a trading decision based on the information? How strong is the correlation when allowing for the confidence factor associated with this many instances? I’m still researching how to quantify...

History may not repeat, but…

Somebody from a private forum I belong to known as “T-Theory” posted a graph that shows the rationale for being long the S&P when the 10 week exponential moving average is trending above the 50 week exponential moving average and being short in the opposite case. T-Theory is an approach invented by the late Terry Landry to viewing market behaviors in terms of cycles – that is markets tend to spend half of their time rising at a faster pace than the other half of the time and that these periods tend to occur symmetrically. For a more detailed explanation of T-Theory, see http://cdn3.traderslaboratory.com/forums/attachments/34/31965d1350013176-beyond-taylor-a1997introttheory_.pdf The below graph shows the exponential moving averages (EMA) for 10 weeks using the red line and for 50 weeks using the green line. A sell signal is generated when the red line crosses the green line while a buy signal occurs when the green line crosses over the red line. A sell signal was recently generated for the S&P 500 and also exists for the other major indexes including the Dow. A sell signal has also been in place for many foreign indexes for many months including the Chinese market. Clearly, this was a good strategy since 2000. I decided to quantify the benefits since then as well as over the longer-term using my equities database. I recently added .NET SQLCLR (A mechanism for Microsoft SQL Server that allows one to write .NET code and integrate into database functions) functionality to my SQL Server database that makes it relatively easy to calculate different technical indicators on the fly. I have been able to build...

A couple of factoids on data security to think about

Here’s factoid that most people, even many at the IT management level don’t realize: A 128 GB Thumb drive which can be had for under $40.00 can store enough information to accomplish identity theft for the population of the entire world (7 billion people). 128 GB is approximately 131 billion bytes which is 19 bytes per person. Name and address data can normally be compressed by a factor of 3 and birth dates and social security numbers only use 6 bytes in packed format. So, figuring about 40 bytes for name and address, this will compress to 19 bytes per person. Another factoid: It takes less than 90 seconds to download 10 million records containing a person’s name, address, spouse, birth date, and social security number. This is based on a cable modem connection of 50 Mb/S which equates to about 6 MB/s or 360 MB/minute. A person’s complete identity record normally is less than 50 bytes. 50 * 10 million so the entire data set for 10 million people is only 500 MB. That is in uncompressed format. With compression, it takes less than 30 seconds. Unfortunately, my experience has been that the government and most companies are not making the effort of protecting data much of a priority. They throw money at it for sophisticated products that do not actually address the problem. They pay for expensive audits from companies that do not actually have the technical expertise to spot them.  The regulatory audits seldom find actual problems as they are focused on outdated security mechanisms that do not have applicability to the most common scenarios whereby...
High-performance Statistical Queries using Self-joins

High-performance Statistical Queries using Self-joins

In my pursuit of understanding asset markets, I’ve maintained a SQL Server database with a lot of information about the stock market and other indexes. Using some data services along with SSIS, this database has been kept current to the point that it now has over 3 billion records in total including 2.3 billion records in an intraday table. Most of my queries and interest concerns cumulative equity and index end-of-day history which is only 75 million row and the options data since 2003 which is now up to 175 million rows. To be able to query this level of data, I utilize Fusion-io PCIE SSD storage for the SQL Server database. Using self-joins can produce some very interesting analysis. For example, the below query outlines the performance of a few global indexes where there have been large bounces close to market tops and the ensuing performance afterwards. This query complete in just a few seconds. There are couple of tricks that make this run faster – one is the storing of a relative day number to avoid performance issues with working around weekends for querying prior dated history. The day number is sequential across holidays and weekends so a direct link can be done without a range test. The other trick is that the table is partitioned based on the date which allows a good deal of parallelism. Here is the query select h.TradingSymbol, h.MarketDate, h.ChgPct + hprev.ChgPct as TwoDayChg, hprev.PriceAtClose as Price, (h20.PriceAtClose – h.PriceAtClose) / h.PriceAtClose as Chg1Month,(h60.PriceAtClose – h.PriceAtClose) / h.PriceAtClose as Chg3Month,(h180.PriceAtClose – h.PriceAtClose) / h.PriceAtClose as Chg9Month,(h360.PriceAtClose – h.PriceAtClose) / h.PriceAtClose as Chg18Month,(h540.PriceAtClose...

Increased data security focus for Authintel

We’ve all seen the news about the latest data security breaches. While bureaucrats blame these on sophisticated hacks from China, the reality is that these are mainly due to negligence and are so simple that a child with basic computer knowledge could pull off many of these. http://www.darkreading.com/attacks-breaches/the-eight-most-common-causes-of-data-breaches/d/d-id/1139795? The problem is that technology has focused on encryption and firewalls while neglecting security at the basic data and application tiers and has very little concept of proactive monitoring of actual user behaviors. Most data theft occurs due to compromised employees or stolen credentials wherein the perpetrator appears to the system as a trusted user and is not monitored. Our company holds credentials that include a PhD for automated learning, the highest-level ISC security certification, CISSP, SQL Server master, and certified .NET application developers. We are uniquely qualified to resolve the use cases that lead to security breaches at the application and data levels. We have produced a video that outlines how millions of PII data records can be stolen without trace in less than 5 minutes that will work at most companies using an ordinary user account. We are focused on resolution of the actual uses cases that lead to data theft rather than on elaborate technologies that are difficult to configure and mostly ineffective. Contact us and we can perform an audit as well as provide remediation including deployment of automated scripts and...

Power BI Experiences

I’ve spent quite a bit of time the last few days trying to get Power BI preview to work for accessing a tabular model so am sharing my experience. Based on that experience, I am making some assertions that are not even in the Microsoft documentation, albeit most of this is found on various other blogs of users who have tried to get this working. Below is a screen snapshot of a demo that my adult son, Blake built in his spare time. My thanks to Blake for his work to do the tabular model and report. My conclusion is that the Power BI configuration is extremely complex, difficult to troubleshoot and requires Active Directory infrastructure configuration. However, once it is working it is seamless to develop reports from a single Power BI site that access tabular models exposed by other organizations without the need to store the data locally. Here are the steps that I had to do – your mileage may vary and I’m happy if you didn’t have to go through all of this:1)    Power BI can only access a tabular model on a machine joined to a domain. 2)    The service account registered with Power BI must be defined in Active Directory with a full UPN exactly matching the Power BI logon.  For example, if the logon account is john.doe@company.onmicrosoft.com, then there must be an account in Active Directory registered with john.doe@company.onmicrosoft.com This is because Power BI sends all requests for tabular data using the “effectiveusername” technique (see profiler trace at end of this post).  This requires the following steps:a.    Use Active Directory Domains and...