How to troubleshoot chart brushing?

Mike Sheehan ●

Hi Stephen,

Thank you for reaching out. I'm wondering whether your Passenger Events by Time report is using Timestamp data while your Events by Day report may be using Date data or simply a text Dimension. This could plausibly cause this issue. Due to the many variations of data possible it's difficult to give highly specific error messages. This one is actually relatively helpful, as it at least tells you where you where to look. The logs could provide some more information here, but of course this is hard to say without better knowing your data.

Probably the best way to test this is by creating the same Day of Week calculated field in the corresponding report, does the filtering work properly?

As a side note, you can also find some more information on chart brushing in our Wiki article on Chart Brushing as well. This Wiki entry features a video called "How to use brushing to filter data in a chart" and lists Limitations and Incompatible charts, which you may find helpful.

Regards,

Mike

Reply URL

1

Stephen Johnson ●

Hi Mike,

I don't understand what you're asking. Both reports are built from the same view. Each record in that view is tagged with a time bucket and a day of week. The former is an integer being formatted as a time of day and the second is an integer being formatted as a date hierarchy part. The only difference between the two reports is that one is aggregating by time bucket and the other is aggregating by day of the week. They each have a filter setup on the metric the other is aggregating on. When I brush on one of the charts, I expect it to simply populate the filter of the other.

I've done some more testing and this is what I've found. The time bucket 5:30am is represented in the data as the integer 19,800 (seconds since midnight). If I go to either report and enter 19,800 in the time bucket filter, the data is filtered correctly. But if I brush the "05:30:00" time bucket on the graph and look at the filter value that gets sent to the events by day chart (by clicking the full screen button), I can see that the filter value being used is 53,000 (when it should be 19,800).

This suggests that Yellowfin is not tracking data conversion correctly. It is using one method to convert the seconds since midnight data to the hours:minutes:seconds format for the chart axis, but when passing the data back to the filter it is not using the correct reverse conversion. Instead it's taking hours:minutes:seconds, stripping out the colons, and casting that string as an integer.

Is there a way to fix this? Or is Yellowfin unable to implement chart brushing for any field with formatting applied?

Reply URL

1

Stephen Johnson ●

Another wrinkle: When I brush 5:30 and go to the other chart and I see that 53,000 is being used as the filter parameter, the error message on that chart is "Invalid Filter Data". If I delete the 53,000 entry in the filter list and manually type 53,000 and run the report again, the message I get is "No results returned".

So maybe Yellowfin is not even trying to pass an integer when I brush?

This is really frustrating.

URL

1

Mike Sheehan ●

Hi Stephen,

It seems the only option I have here is to attempt to replicate this on my end, as there's really no way to say what's going on without having some data to approximate your setup here. In order to do this I'm going to require more detailed replication steps with corresponding screenshots, so I can get a better sense of how your reports, filters, and dashboard is setup so I can match them as close as possible, so if you could provide this info that'd be great.

Can you also please send where you are seeing the data in seconds? Is this in the SQL? Regardless of this, can you also please send the SQL statements being generated on the reports that are being filtered? This should provide insight into what is actually being queried when you Keep or Exclude results via brushing.

I think the logs could also be helpful here. Please replicate the issue where you're seeing "Invalid Filter Data" then compress and attach your entire logs folder located at <YellowfinInstall>/appserver/logs to your response here.

Thanks,

Mike

Reply URL

1

Stephen Johnson ●

Why don't we just set up a screen share? That would take way less of both of our time than trying to replicate my setup.

URL

1

Mike Sheehan ●

Hi Stephen,

That should indeed give me the info I require more quickly! What time works best for you? I'm generally available 9AM - 4PM MT M-F, but today won't be available again until about about 3 PM. I think you're in VA, so perhaps some time tomorrow morning?

Regards,

Mike

URL

1

Stephen Johnson ●

Hi Mike. Let's meet this morning. Do you have access to my account email address? Email me when you get in and I will send you a join.me link for screen sharing.

URL

1

Mike Sheehan ●

Hi Stephen,

Do you have RingCentral Meetings? That's what we typically use, then I'd send you a meeting link on here. If you don't have access to that I'll download a trial of join.me. Does 9 work?

Thanks,

Mike

URL

1

Stephen Johnson ●

If you are initiating the meeting then RingCentral is fine. 9 am MT works for me.

URL

1

Mike Sheehan ●

Hi Stephen,

Excellent:

Mike Sheehan is inviting you to a RingCentral meeting.

Join from PC, Mac, iOS or Android: https://meetings.ringcentral.com/j/1480959451

Or iPhone one-tap: +1(773)2319226,,1480959451#

Or Telephone: Dial: +1 (773) 231 9226

Meeting ID: 148 095 9451

International numbers available: https://meetings.ringcentral.com/teleconference

URL

1

Mike Sheehan ●

Hi Stephen,

I was able to replicate this issue and have raised a defect for this, for which I'll post any related updates here. That said, this setup does not seem ideal for what you're trying to accomplish. Formatting an integer into a Timestamp is bound to cause unexpected behavior. While this may technically be defective behavior, the overall setup and formatting here is not recommended and if the Time data was actually a Timestamp you'd likely not be encountering this error. I was informed by your account manager you have a call with consulting later today. They are going to discuss this ticket with you further then.

On the note of the Numeric Display chart, I'm posting further information in the corresponding ticket (8345) to keep better track of the issue.

Also as mentioned on our call, this is the link to download latest versions & builds: https://portal.yellowfinbi.com/yf_latestbuild.jsp

And finally, I've created an Idea ticket linked to your name to add the ability to execute upgrades through the Admin Console, which you can reference here for potential future updates.

Regards,

Mike

Reply URL

1

Stephen Johnson ●

There was a miscommunication. My call this afternoon was only to get a tour of the support resources (which, as you know, I am already quite familiar with!).

I am open to any suggestions you may have about how to bring data into Yellowfin in a better way. However, we are building on top of a DB of another software platform, so I cannot change the way our data is formatted in that DB. Dates in particular have been a challenge. Our dates are stored as integers in our DB and working with those dates has been difficult with every BI suite I have tested.

URL

1

Mike Sheehan ●

Hi Stephen,

Ashley just informed me of that. Hopefully we can get that consulting call setup in the next couple of weeks. I do think that'd be beneficial. They'll have some idea on how to better set up your data for sure. As for the way the date data is currently formatted... eesh. Haha. That all makes sense. There are many places where YF is going to treat integers differently than it would Date data. I'm wondering if some sort of Freehand SQL CAST statement could translate the integers to be treated as actual date data. This too would fall under consulting, or be something to figure out in-house, but I'm thinking that may be a possibility. The thing is even if this defect is fixed, it's not going to help with these reports or dashboard, as it's still going to show No Results Returned, because choosing 53000 as the filter value, or pretty much any of the other converted "Time" data, isn't actually a value in the Passenger Events by Day table. The way these reports are setup and filtered, as well as the underlying data itself will likely all need to be changed for this to work in the way you're wanting it to and this type of advanced content creation and setup is all more apart of the consulting realm, so hopefully some of this can get sorted there. Wish there was an easier way to have this happen for the time being!

Regards,

Mike

URL

1

Stephen Johnson ●

Thanks for the feedback Mike.

I disagree with your assessment that what I'm trying to do is advanced. Yellowfin should have been programmed to distinguish between underlying data being used to aggregate queries and formatted data used to display those groupings on a chart axis. When you brush on a chart it should obviously pass the underlying data to the appropriate filter, not the formatted chart axis label.

An analog would be this: Imagine you have a web app where you are having users chose a product from a dropdown list that they would like to order. In the dropdown list you put the nicely formatted, descriptive names of the products so that it is user friendly. But when they make a selection, you program the thing to send a product ID back to the server, not the formatted, user-friendly name. This is more performant on the back end (integer comparison vs string comparison) and more resilient to changes to product names, etc.

It's the same thing here. Yellowfin should display the user-friendly, formatted chart label to the user and when they brush on that chart, it should grab the underlying data to pass to the designated filter. If I decide tomorrow that I want to change the way my chart-axis are formatted and displayed to the user, that should not break my brushing feature.

This is especially evident when you look at how the Yellowfin filters were programmed. When you and I were working together on the full-screen report looking at the "Events by Day" and manually entering the desired time bucket, we had to enter the integer format because that's the only thing the filter will accept, despite the fact that the integer is formatted as a time at the view level. The filter accepts only the raw, unformatted data as input. Period. And yet your programmers programmed the brush feature to pass it a formatted chart axis.

To go back to our analog, that's like having the front-end developers program the website to send back the formatted product title when the back-end developers programmed the server to expect a product ID. Obviously that software won't work. And that's what we're seeing here.

Thanks for the feedback Mike.

I disagree with your assessment that what I'm trying to do is advanced. Yellowfin should have been programmed to distinguish between underlying data being used to aggregate queries and formatted data used to display those groupings on a chart axis. When you brush on a chart it should obviously pass the underlying data to the appropriate filter, not the formatted chart axis label.

An analog would be this: Imagine you have a web app where you are having users chose a product from a dropdown list that they would like to order. In the dropdown list you put the nicely formatted, descriptive names of the products so that it is user friendly. But when they make a selection, you program the thing to send a product ID back to the server, not the formatted, user-friendly name. This is more performant on the back end (integer comparison vs string comparison) and more resilient to changes to product names, etc.

It's the same thing here. Yellowfin should display the user-friendly, formatted chart label to the user and when they brush on that chart, it should grab the underlying data to pass to the designated filter. If I decide tomorrow that I want to change the way my chart-axis are formatted and displayed to the user, that should not break my brushing feature.

This is especially evident when you look at how the Yellowfin filters were programmed. When you and I were working together on the full-screen report looking at the "Events by Day" and manually entering the desired time bucket, we had to enter the integer format because that's the only thing the filter will accept, despite the fact that the integer is formatted as a time at the view level. The filter accepts only the raw, unformatted data as input. Period. And yet your programmers programmed the brush feature to pass it a formatted chart axis.

To go back to our analog, that's like having the front-end developers program the website to send back the formatted product title when the back-end developers programmed the server to expect a product ID. Obviously that software won't work. And that's what we're seeing here.

URL

1

Mike Sheehan ●

Hi Stephen,

Thanks for your response. That's a helpful way of describing it. I spoke with my colleagues here and we are all in agreement you are right here. It should be filtering the underlying data's value and not the formatted value. I've made specific note of this in our internal task. I'll keep you posted.

Regards,

Mike

URL

1

Stephen Johnson ●

Cheers.

URL

1

Mike Sheehan ●

Cheers! We'll keep you posted.

Regards,

Mike

URL

How to troubleshoot chart brushing?

Replies (4)